Physical Intelligence, the San Francisco-based startup that has raised more than $400 million, has open-sourced its Pi0 robotic foundation model. Pi0 was introduced a few months ago and can be tuned to a range of tasks, including folding laundry, cleaning a table, scooping coffee beans, and more.
Physical Intelligence has released the code and weights for Pi0 as part of its experimental openpi repository on GitHub. It also provides checkpoints for a few “simple tasks” on available platforms such as ALOHA and DROID, example code to run inference on real-world and simulated robot platforms, and the code for fine-tuning the base π0 model for your own tasks and platforms.
The company said between 1-20 hours of data was sufficient to tune Pi0 to a variety of tasks in its own experiments. HuggingFace has also prepared a PyTorch port of openpi for those developers who prefer PyTorch over JAX.
“We believe that general purpose models that can control any robot to perform any task will be the future of robotic control,” the company said. However, there are many unanswered questions, both in how such models can be built, and in how they will be used, adapted, and deployed. We believe that with π0, we’ve taken an important step forward, but some of the most difficult challenges are in the future.
“To develop truly general and capable models, the robotics community will need to work together, and our aim with releasing openpi is to contribute to this shared effort. In the same way that effective open-source language models (LLMs) and vision-language models (VLMs) have led to a Cambrian explosion of new LLM and VLM applications, new methods in research, and new products, we hope that openpi will lead to new and creative uses of robotic foundation models, public sharing of larger and more exciting datasets, and new technologies.”
Now accepting session submissions!
Features of openpi
Physical Intelligence is providing sample code that can be used to implement a client for your own robot. Below are some highlights of the pre-trained checkpoints provided in openpi. Check out the openpi repository documentation for a complete list.
Pi0 base: This is the standard pre-trained model. The model is trained on OXE and 7 of Physical Intelligence’s robot platforms. It is designed for fine-tuning, though it can be used in zero shot for tasks that are present in the pre-training data.
Pi0-FAST base: This model uses the FAST tokenizer to enable control via autoregressive discretization. Physical Intelligence said it provides somewhat better language following performance, but has higher inference cost (about 4-5x higher). This is a good choice if you prefer to use discretization rather than flow matching, according to the company.
Pi0-FAST DROID and π0 DROID: openpi provides two models that are fine-tuned to the DROID dataset, which consists of diverse tasks in diverse environments with a Franka robotic arm. These are the first models that are able to follow instructions successfully in entirely new environments with DROID platforms, according to Physical Intelligence.
Pi0 ALOHA: A set of checkpoints fine-tuned for tasks such as towel folding, food scooping, and more on the ALOHA platform. These checkpoints might be quite sensitive to overall robot setup, the company said, though it was able to run them on an entirely new ALOHA station that did not appear in the training data.
Pi0 Libero: This checkpoint is fine-tuned for the Libero benchmark, and can be evaluated on Libero tasks out of the box.
“Our aim with this release is to enable anyone to experiment with fine-tuning π0 to their own robots and tasks,” Physical Intelligence said. “We believe that these generalist robot policies hold the potential to not only enable effective robotic learning, but in the long run transform how we think about artificial intelligence: in the same way that people possess cognitive abilities that are grounded in the physical world, future AI systems will be able to interact with the world around them, understand physical interactions and processes at an intuitive level, and reason about cause and effect. We believe that embodiment is key to this, and by making π0 available to everyone, we hope to contribute to progress toward broadly capable and general-purpose physical intelligence.”
Learn about foundation models at Robotics Summit & Expo
The promise of foundation models is to give robots the ability to generalize actions from fewer examples than traditional AI techniques. A number of companies have popped up recently to work on robotics foundation models, including Pittsburgh-based Skild AI. Existing companies such as Ambi Robotics, Cobot, Figure AI and others are developing their own foundation models to deploy in specific applications.
Daniela Rus, director of MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), is delivering the opening keynote at the Robotics Summit & Expo, which runs April 30-May 1 in Boston. Her keynote will explore physical intelligence, which she said is achieved when AI’s power to understand text, images, signals, and other information is used to make physical machines, such as robots, intelligent. Rus’ keynote will discuss the challenges of transformer-based foundational AI models. She will also introduce alternative physics-based models and explain how they achieve performance efficiently.
Produced by The Robot Report, the Robotics Summit & Expo brings together 5,000+ developers focused on building robots for a variety of commercial industries. Attendees will gain insights into the latest enabling technologies, engineering best practices, and emerging trends. There will be 70-plus speakers on stage, 10+ hours of dedicated networking time, a Women in Robotics Breakfast, a career fair, startup showcase, and more. Returning to the show is the RBR50 Pavilion and RBR50 Awards Dinner that will honor the winners of the annual RBR50 Robotics Innovation Awards.
Daniela Rus, director of MIT CSAIL.
Tell Us What You Think!