Choreographed integration of human workers, robotic and agentic systems, and equipment in a facility digital twin. | Source: Accenture, KION Group.
NVIDIA Corp.’s CEO Jensen Huang made a slew of announcements yesterday at CES 2025 in Las Vegas. They included the Mega Omniverse blueprint for building industrial robot fleet digital twins, adding generative physical AI to Omniverse, launching the Cosmos World Foundation Model platform, and releasing updates to its Isaac platform.
These announcements showed how NVIDIA is doubling down on investing in artificial intelligence technologies, particularly generative AI, for robotics. In addition to its new products, the Santa Clara, Calif.-based company announced that Toyota, Aurora, and Continental are developing their consumer and commercial vehicle fleets with NVIDIA computing and AI.
NVIDIA also said its DRIVE Hyperion Platform has achieved critical automotive safety and cybersecurity milestones. It said the platform has passed industry-safety assessments by TÜV SÜD and TÜV Rheinland — two industry authorities for automotive-grade safety and cybersecurity.
The company’s “end-to-end” system includes the DRIVE AGX system-on-a-chip (SoC) and reference board design, the NVIDIA DriveOS automotive operating system, a sensor suite, and an active safety and SAE Level 2+ driving stack.
NVIDIA updates Omniverse
NVIDIA announced Mega, an Omniverse blueprint for developing, testing, and optimizing physical AI and robot fleets at scale in digital twins before deployment into real-world facilities.
Mega offers enterprises a reference architecture of its accelerated computing, AI, NVIDIA Isaac, and NVIDIA Omniverse technologies, the company said. This allows them to develop and test digital twins for testing AI-powered “brains” that drive robots, video analytics, AI agents, equipment, and more.
NVIDIA added that the new Omniverse framework can handle enormous complexity and scale. It can bring software-defined capabilities to physical facilities, enabling continuous development, testing, optimization, and deployment, claimed the company.
With Mega-driven digital twins, including a world simulator that coordinates all robot activities and sensor data, enterprises can continuously update robots for intelligent routes and tasks for operational efficiencies, NVIDIA said.
The blueprint uses Omniverse Cloud Sensor RTX application programming interfaces (APIs), which enable developers to render data from any type of intelligent machine in the factory, simultaneously, for high-fidelity large-scale sensor simulation. This allows robots to be tested in an infinite number of scenarios within the digital twin, using synthetic data in a software-in–the-loop pipeline with NVIDIA Isaac ROS.
NVIDIA also announced generative AI models and blueprints to expand NVIDIA Omniverse integration further into physical AI applications such as robotics, autonomous vehicles (AVs), and vision AI. The company said these models accelerate each step of creating 3D worlds for physical AI simulation, including world-building, labeling the world with physical attributes, and making it photoreal.
“Physical AI will revolutionize the $50 trillion manufacturing and logistics industries. Everything that moves — from cars and trucks to factories and warehouses — will be robotic and embodied by AI,” stated Huang. “NVIDIA’s Omniverse digital twin operating system and Cosmos physical AI serve as the foundational libraries for digitalizing the world’s physical industries.”
Cosmos world foundation model aims to accelerate physical AI development
Companies including 1X, Agile Robots, Agility, Figure AI, Foretellix, Fourier, Galbot, Hillbot, IntBot, Neura Robotics, Skild AI, Uber, Virtual Incision, Waabi, and XPENG are among the first to adopt Cosmos. | Source: NVIDIA
In addition to its Omniverse updates, NVIDIA also released Cosmos, a platform composed of generative world foundation models, advanced tokenizers, guardrails, and an accelerated video processing pipeline built to advance the development of physical AI systems such as AVs and robots.
The company asserted that physical AI models are costly to develop and require vast amounts of real-world data and testing. Cosmos world foundation models, or WFMs, offer developers an easy way to generate massive amounts of photoreal, physics-based synthetic data to train and evaluate their existing models. Developers can also build custom models by fine-tuning Cosmos WFMs.
NVIDIA noted that Cosmos’ suite of open models enables developers to customize the WFMs with datasets, such as video recordings of AV trips or robots navigating a warehouse, according to the needs of their target applications.
The company said it designed its WFMs for physical AI research and development. The WFMs can generate physics-based videos from a combination of inputs, like text, image, and video, as well as robot sensor or motion data.
“The ChatGPT moment for robotics is coming. Like large language models, world foundation models are fundamental to advancing robot and AV development, yet not all developers have the expertise and resources to train their own,” Huang said. “We created Cosmos to democratize physical AI and put general robotics in reach of every developer.”
NVIDIA said Cosmos models will be available under an open model license to accelerate the work of the robotics, AI, and AV community. Developers can preview the first models on the NVIDIA API catalog, or download the family of models and fine-tuning framework from the NVIDIA NGC catalog or Hugging Face.
NVIDIA also revises Isaac
NVIDIA Isaac is a platform of accelerated libraries, application frameworks, and AI models that it said can accelerate the development of AI robots. It’s made up of four different applications: Isaac Sim, Isaac Lab, Isaac Manipulator, and Isaac Perceptor.
NVIDIA Isaac Sim is a reference application built on NVIDIA Omniverse that enables users to develop, simulate, and test AI-driven robots in physically based virtual environments. Isaac Sim 4.5 will offer a number of significant changes, including the following:
- A reference application template
- Improved Unified Robot Description Format (URDF) import and setup
- Improved physics simulation and modeling
- New joint visualization tool
- Simulation accuracy and statistics
- NVIDIA Cosmos world foundation model
NVIDIA Isaac Lab is an open-source unified framework for robot learning to train robot policies. Isaac Lab is built on top of NVIDIA Isaac Sim, helping developers and researchers more efficiently build intelligent, adaptable robots with robust, perception-enabled, simulation-trained policies. The updated version of Isaac Lab includes performance and usability improvements like tiled rendering and other quality-of-life improvements.
NVIDIA Isaac Manipulator, built on ROS 2, is a collection of NVIDIA CUDA-accelerated libraries, AI models, and reference workflows. It now includes new end-to-end reference workflows for pick-and-place and object following, enabling users to quickly get started on fundamental industrial robot arm tasks, including object-following and pick-and-place.
Finally, NVIDIA Isaac Perceptor, also built on ROS 2, is a collection of libraries, models, and reference workflows for the development of autonomous mobile robots (AMRs). It enables AMRs to perceive, localize, and operate in unstructured environments such as warehouses or factories.
NVIDIA said its latest updates bring significant improvements to AMR environmental awareness and operational efficiency in dynamic settings. They include a new end-to-end visual simultaneous localization and mapping (SLAM) reference workflow, new examples of running nvblox with multiple cameras for 3D scene reconstruction with people detection and dynamic scene elements, and improved D scene reconstruction by running Isaac Perceptor on multiple RGB-D cameras.
Save now with early bird discount
Tell Us What You Think!