Immervision, a provider of optical and imaging technology, today announced JOYCE, a humanoid robot and development kit. The Montreal-based company said it hopes the robotics and computer vision community will help upgrade JOYCE and create new applications “not possible before by humans or other robots” in a series of international challenges.

For 20 years, Immervision has developed and licensed wide-angle lenses and imaging software for artificial intelligence and machine vision. The company, which has about 25 staffers, said its technology is used in the aerospace, automotive, robotics, security, and consumer product industries, among others.

“We are the largest optical design firm not attached to a lens manufacturer,” said Allesandro Gasparini, executive vice president of operations and chief commercial officer of Immervision. “We innovate on the optical and image-processing side, ‘from glass to glass,’ meaning from the lens to the display of data.”

“We take experience from some industries and apply them to others, and we license patents out,” he told The Robot Report. “We work with companies such as Qualcomm and Intel.”

Development kit comes with cameras

“We are excited to launch the industry’s first collaborative computer vision humanoid robot,” stated Pascale Nini, president and CEO of Immervision. “[It is] designed for intelligent vision, with 360-degree, spherical field-of-view achieved, thanks to the most advanced freeform panomorph lens design, data-in-picture technology for fusing data from multiple sensors with vision in real-time, image-processing algorithms, APIs and a programming interface from which to add other sensors and software like MEMS, microphones, ultrasound, and additional machine learning and AI algorithms.”

JOYCE includes three ultra-wide-angle panomorph cameras calibrated to give 2D hemispheric, 3D stereoscopic hemispheric, or full 360 x 360 spherical capture and viewing of the environment, said Immervision.

The robot’s data-in-picture technology enables each video frame to be enriched with data from a wide array of sensors. These can provide contextual information to AI and neural networks, computer vision, and simultaneous localization and mapping (SLAM) algorithms to help increase her visual perception, said the company.

The development kit is available to developers, universities, and technology companies to add additional sensors, software, and algorithms to enhance JOYCE’s perception and understanding of her environment to solve computer vision challenges.

“We cannot wait to collaborate with the brightest minds in computer vision and AI to explore bleeding-edge solutions to industry-specific challenges,” said Nini. “When you go to pick up something, you may not realize that your sense of touch is connected to sight. JOYCE is connected to the Internet, and the goal is to focus on community development.”

Challenge and use cases for JOYCE

Immervision is encouraging members of the computer vision community to add their technologies to upgrade JOYCE in a series of international challenges to be launched in 2021. The company is also looking for partners.

“For example, one Immovision partner is providing a time-of-flight sensor, and we’re looking to add a microprocessor,” said Alain Paquin, a longtime entrepreneur and head of the JOYCE project at Immervision. “We’ll also have microphones and other sensors prior to the first challenge.”

“Unlike other branded robots, we’re asking the community to find the best solutions as we look to the creation of new intellectual property and cross-pollination among industries,” said Gasparini. “I got the latest Roomba and was excited to watch it go around. While it has many sensors and SLAM, it still doesn’t understand when it hits a chair leg and needs to find a path around it.”

“We’re looking to find the best way to perceive more smartly,” he added. “Humans can see in 210 degrees, but robots will be able to see 360 degrees, with 3D image reconstruction and 180-degree stereographic vision. Developers could also add thermal or gas-analysis sensors.”

“Challenge participants will be able to share their applications, and the winners will get their sensors mounted in the actual robot,” said Paquin. “JOYCE is humanoid as a hub to bring solutions. It’s a way to personify technology for a machine to understand what it sees. We’re looking for engineers to combine perception for healthcare, mobility, and the Internet of things.”

Will the humanoid robot be able to walk around using vision? “For now, the first thing is to move her head,” acknowledged Paquin. “Down the road, she will walk, but that will be done with partners.”

JOYCE could lead to advances in smart home devices, autonomous vehicles, and precision agriculture, according to Immervision. It could also improve medical diagnostics and help firefighters find people through smoke, said the company.

To learn more about JOYCE, visit Immervision’s website.