Fraunhofer taps NVIDIA’s simulation skills to improve robot designs

The Fraunhofer Institute in Germany conducts practical research in a number of important fields, including AI, cybersecurity, and medicine. One of its 76 research institutes, the Fraunhofer IML group, seeks to advance robotics and logistics. The researchers are testing the simulation capabilities of NVIDIA Isaac Sim to potentially enhance the design of robots. 

Its most recent mobile robot development, O3dyn, uses technologies developed by NVIDIA for simulation and robotics to produce an indoor-outdoor autonomous mobile robot (AMR).

“We’re looking at how we can go as fast and as safely as possible in logistics scenarios,” said Julian Eber, a robotics and AI researcher at Fraunhofer IML.

The latest research aims to bridge the sim-to-real gap by developing and validating these AMRs in simulation. The researchers use Isaac Sim to train the AMR virtually by placing it in photorealistic, physically correct 3D environments.

The team leverages Isaac SIM to test the operation of O3dyn by importing a complete 3D CAD model of the robot into the simulated environment. The accurate specifications are combined with Omniverse PhysX to deliver a realistic response to any input. The robot uses simulated warehouse environments to train the AI guidance on how to navigate and operate in its intended missions.

In the physical world, O3dyn uses a variety of camera and sensor inputs into the NVIDIA Jetson edge AI and robotics platform to aid with navigation. With improved speed and agility, it can move at up to 13.4 m/s (30 mph) and has omnidirectional wheels that can move in any direction.

“The omnidirectional dynamics is very unique, and there’s nothing like this that we know of in the market,” said Sören Kerner, head of AI and autonomous systems at Fraunhofer IML.

As a result, the virtual robot can move as quickly in a simulation as the actual robot can in the physical world. Harnessing the virtual environment allows Fraunhofer to accelerate development, safely increase accuracy for real-world deployment and scale up faster.

Fraunhofer refers to this idea as simulation-based AI. Fraunhofer is making the AMR simulation model open source so that developers can enhance it in order to achieve results more quickly.