Memories.ai models label every clip: Memories-H0 for hand pose, EgoAction for action segments. | Credit: Memories.ai
Memories.ai is building a data pipeline for robotics companies that combines standardized video collection with AI-powered annotation to turn everyday egocentric footage into training data for robots.
The San Francisco-based company is developing software and AI to help robots understand the physical world. It builds video-understanding models and data infrastructure, and it collects and annotates first-person (egocentric) video to provide high-quality training data to robotics companies.
The longer-term aim is to be a data-infrastructure partner for robotics and physical AI, not just a data vendor.
Memories.ai builds platform on video data
Memories.ai claimed that its video data infrastructure platform is built for physical AI, robotics, world models, vision-language-action (VLA) models, and humanoids. It provides a curated video data lake featuring egocentric video, along with custom annotation services and dataset tools to train robot learning models.
The company obtains egocentric video through data collectors and partners, providing them with standardized capture hardware—described as a modular camera setup—and uniform process to collect footage in a consistent way.
Memories.ai then processes and annotates the data with its models and makes it available through its data infrastructure. The company has published multiple elements of its research on its GitHub repo. It has also published at the ICLR, NeurIPS, and EMNLP events.
Submit your innovation now!NVIDIA Cosmos helps realize physical AI vision
“Our vision right now is to accelerate physical intelligence, and our vision is to stand really at the frontier to being able to be part of the robotics game,” Shawn Shen, co-founder and CEO of Memories.ai, recently told The Robot Report. “We really bring robotics into the actual physical world, and I believe that’s our whole team’s vision. And whatever can help us to make that vision come faster, we’re happy to do it.”
Shen said the company is focused on processing its own data, rather than collecting video from a wide range of devices and sources. Once it gets data from partners, Memories.ai processes and annotates the footage itself.
The company said last month that it has used NVIDIA Cosmos to interpret video entering its data lake. It then curates that data to post-train Cosmos for customers’ robotics and industrial applications. This loop turns real-world video into models tailored to the tasks our customers need to solve.
Pipeline has a modular design
Memories.ai said its setup is modular to support diverse tasks and camera views. Shen asserted that the company wants to grow with the industry as AI models mature for more robotics applications.
“We want to be long-term partners,” he said. “We don’t want to just sell you the data. In the future, we want to work with you on the models. We want to work with you on the infra because we know that you are, and will be, limited by your resources too.”
For robotics customers, Memory AI plans to tailor the data to their needs, offer exclusive or non-exclusive arrangements, and potentially expand its role into models and video infrastructure.
Tell Us What You Think!