The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe

Waymo dataset to crowdsource behavior prediction breakthroughs

By Steve Crowe | March 11, 2021

Waymo Open Dataset behavior prediction

The Waymo Open Dataset contains 570-plus hours of unique motion behavior data. | Photo Credit: Waymo

Developing autonomous vehicles has turned out to be much harder than originally thought. Companies across the board, but especially those developing robotaxis, have universally missed the commercialization dates they once predicted.

One of the hardest problems in autonomous driving, according to Waymo, is accurate behavior prediction of other road users. While progress has certainly been made, Waymo and other autonomous vehicle companies are waiting for new breakthroughs to occur in motion forecasting.

To help push things along, the 2020 RBR50 company expanded the Waymo Open Dataset with a new motion dataset. Waymo claimed its “the largest interactive dataset yet released for research into behavior prediction and motion forecasting for autonomous driving.” The motion dataset offers object trajectories and corresponding 3D maps for over 100,000 segments, each 20 seconds long and mined for interesting interactions with cars, people, and more. The dataset contains 570-plus hours of data captured by Waymo’s vehicles.

Waymo said the dataset features a variety of road types and driving conditions from urban environments in San Francisco, Phoenix, Mountain View, Los Angeles, Detroit and Seattle. You can read more about the new motion dataset on Waymo’s GitHub repo or download the dataset here. Waymo also released a paper about the dataset.

“We have mined the data specifically to include interesting examples of agents interacting – whether it’s cyclists and vehicles sharing the roadway, cars quickly passing through a busy junction, or groups of pedestrians clustering on the sidewalk,” said Drago Anguelov, distinguished research scientist, Waymo. “At 20 seconds, each segment is long enough to train models that can capture complex behaviors. We have also included map information for each scene to provide semantic context that’s important for accurate predictions.”

Oliver Cameron, CEO and co-founder of autonomous vehicle startup Voyage, wrote about prediction being the next leap in self-driving vehicles. In the piece, he explained why prediction has overtaken perception as the biggest challenge in the field. He said “the problem of prediction is central to the most difficult instantiations of the unprotected left turn.”

Other leading autonomous vehicle companies have also released datasets. Aptiv first released its nuScenes dataset in 2018, initially creating it using data collected from 1,000 urban street scenes in Boston and Singapore. Cruise has released AV visualization tools to the public, and Argo AI released two dataset of HD maps called the “Argoverse.” One of those datasets includes “324,557 interesting vehicle trajectories extracted from over 1000 driving hours.”

Waymo San Francisco

A Waymo robotaxi being tested in San Francisco. | Photo Credit: Waymo

Waymo is also launching an open dataset challenge to encourage research teams in their work on behavior and prediction. There are four challenges: motion prediction, interaction prediction, real-time 3D detection, and real-time 2D detection. The winner of each challenge will receive $15,000, with second-place teams receiving $5,000 and third place $2,000.

“We hope this data will be useful for researchers working on behavior prediction in a wide variety of domains and industries,” said Anguelov. He added, “I know first-hand how incredibly important high-quality data is to tackling thorny AV research questions, and I’m incredibly proud of the Waymo research team’s efforts to continue evolving the dataset and making it applicable for an increasing number of research areas. We welcome community feedback on how to make our dataset even more impactful in future updates.”

Waymo recently began limited testing of its robotaxis in San Francisco. The Alphabet subsidiary is proceeding cautiously, using employee volunteers at this point to gather feedback about the technology’s performance. Waymo said that while it does plan to grow its presence there over time, “we don’t have any specific timelines to share about when (or where) we’ll be offering a public service.” San Francisco is also home to Cruise.

Waymo and Cruise were the leaders in the clubhouse according to the California’s Department of Motor Vehicles annual Disengagement Report that came out earlier this month. The report captured the number of times a human driver takes over control of an autonomous driving system being tested on the state’s public roads between December 2019 to November 2020.

About The Author

Steve Crowe

Steve Crowe is Executive Editor, Robotics, WTWH Media, and chair of the Robotics Summit & Expo and RoboBusiness. He is also co-host of The Robot Report Podcast, the top-rated podcast for the robotics industry. He joined WTWH Media in January 2018 after spending four-plus years as Managing Editor of Robotics Trends Media. He can be reached at scrowe@wtwhmedia.com

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

A graphic for Gemini Robotics On-Device.
Google DeepMind introduces on-device Gemini AI model for robots
The Gen3 4NE1 humanoid from NEURA Robotics on display at Automatica 2025.
NEURA Robotics launches latest cognitive robots, Neuraverse ecosystem
NEXCOM is working with NVIDIA on AI robot safety.
NEXCOM NexCOBOT unit joins NVIDIA Halos AI Systems Inspection Lab
Developers of the humanoids and cobots shown here can use NVIDIA's safety platform.
NVIDIA Isaac, Omniverse, and Halos to aid European robotics developers

RBR50 Innovation Awards

“rr
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for Robotics Professionals.
The Robot Report Listing Database

Latest Episode of The Robot Report Podcast

Automated Warehouse Research Reports

Sponsored Content

  • How to Set Up a Planetary Gear Motion with SOLIDWORKS
  • Sager Electronics and its partners, logos shown here, will exhibit at the 2025 Robotics Summit & Expo. Sager Electronics to exhibit at the Robotics Summit & Expo
  • The Shift in Robotics: How Visual Perception is Separating Winners from the Pack
  • An AutoStore automated storage and retrieval grid. Webinar to provide automated storage and retrieval adoption advice
  • Smaller, tougher devices for evolving demands
The Robot Report
  • Automated Warehouse
  • RoboBusiness Event
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Contact Us

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe