Organization: NVIDIA
Country: U.S.
Website: nvidia.com/en-us
Year Founded: 1993
Number of Employees: 500+
Innovation Class: Technology
Robots operating in dynamic, real-world settings must be able to learn new skills quickly and react to their environments. However, a gap between perception and action plus the challenge of transferring skills across contexts can limit robot adaptability, said NVIDIA Corp. To address those limitations, the company has developed NVIDIA Isaac Lab.
Physical AI models enable robots to conduct complex tasks such as dexterous manipulation or locomotion across rough terrain. These would be difficult to program without massive amounts of data and simulation, according to NVIDIA.
“We wanted to enable the industry and developers who wanted to go beyond pre-programmed robots to build specialized robot policies for advanced robotic embodiment,” said NVIDIA. “With Isaac Lab's modular approach, lightweight framework, and perception-in-the-loop capabilities, one can teach robot new skills and train robot policies at scale in simulation for multiple robot instances in one go using NVIDIA tools and technologies.”
Isaac Lab supports both imitation learning and reinforcement learning (RL) to make it easier for developers to efficiently train robots. They can add or update skills depending on changing business needs.
Developers can design, test, and train autonomous machines in a virtual environment with NVIDIA Isaac Sim, a reference application built on NVIDIA Omniverse. Isaac Lab is an open-source robot learning framework built on Isaac Sim to simplify common workflows. It was announced along with Project GR00T in March 2024 and is the successor to NVIDIA Isaac Gym.
The framework includes accurate physics simulation using PhysX, tiled application programming interfaces (APIs) for vectorized rendering, domain randomization, and support for running in the cloud. It also includes “batteries included” support for prebuilt environments such as factories and warehouses, as well as digital twins of different types of robots, including quadrupeds, drones, and humanoids.
Isaac Lab incorporates features from Orbit, an open-source framework jointly developed by NVIDIA, the AI Institute, ETH Zurich, and the University of Toronto. It supports different physics engines and can scale frames per second with multiple graphics processing units (GPUs) training on Linux using the PyTorch distributed framework.
NVIDIA said Isaac Lab can achieve higher performance levels for running thousands of parallel scenario. The cloud-native OSMO orchestration platform coordinates data generation, model training, and software/hardware-in-the-loop workflows across distributed environments.
Isaac Lab 1.0, released in July 2024, added modular capabilities such as a large language model (LLM)-to-reward function reference example for RL and flexible task-design workflows. In November, NVIDIA announced general availability of Version 1.2.
Users of Isaac Lab include Field AI, Mentee Robots, ORBIT-Surgical, and Skild AI. Humanoid robot developers using Isaac Lab include 1X Technologies, Agility Robotics, Berkeley Humanoid, Boston Dynamics, Fourier Intelligence, Galbot, Unitree Robotics, and XPENG Robotics.
The NVIDIA GR00T research team is also using NVIDIA Isaac Lab for robot foundation model training. The framework is an enabler for the next generation of humanoid robot capabilities.