Waymo dataset to crowdsource behavior prediction breakthroughs

Developing autonomous vehicles has turned out to be much harder than originally thought. Companies across the board, but especially those developing robotaxis, have universally missed the commercialization dates they once predicted.

One of the hardest problems in autonomous driving, according to Waymo, is accurate behavior prediction of other road users. While progress has certainly been made, Waymo and other autonomous vehicle companies are waiting for new breakthroughs to occur in motion forecasting.

To help push things along, the 2020 RBR50 company expanded the Waymo Open Dataset with a new motion dataset. Waymo claimed its “the largest interactive dataset yet released for research into behavior prediction and motion forecasting for autonomous driving.” The motion dataset offers object trajectories and corresponding 3D maps for over 100,000 segments, each 20 seconds long and mined for interesting interactions with cars, people, and more. The dataset contains 570-plus hours of data captured by Waymo’s vehicles.

Waymo said the dataset features a variety of road types and driving conditions from urban environments in San Francisco, Phoenix, Mountain View, Los Angeles, Detroit and Seattle. You can read more about the new motion dataset on Waymo’s GitHub repo or download the dataset here. Waymo also released a paper about the dataset.

“We have mined the data specifically to include interesting examples of agents interacting – whether it’s cyclists and vehicles sharing the roadway, cars quickly passing through a busy junction, or groups of pedestrians clustering on the sidewalk,” said Drago Anguelov, distinguished research scientist, Waymo. “At 20 seconds, each segment is long enough to train models that can capture complex behaviors. We have also included map information for each scene to provide semantic context that’s important for accurate predictions.”

Oliver Cameron, CEO and co-founder of autonomous vehicle startup Voyage, wrote about prediction being the next leap in self-driving vehicles. In the piece, he explained why prediction has overtaken perception as the biggest challenge in the field. He said “the problem of prediction is central to the most difficult instantiations of the unprotected left turn.”

Other leading autonomous vehicle companies have also released datasets. Aptiv first released its nuScenes dataset in 2018, initially creating it using data collected from 1,000 urban street scenes in Boston and Singapore. Cruise has released AV visualization tools to the public, and Argo AI released two dataset of HD maps called the “Argoverse.” One of those datasets includes “324,557 interesting vehicle trajectories extracted from over 1000 driving hours.”

Waymo is also launching an open dataset challenge to encourage research teams in their work on behavior and prediction. There are four challenges: motion prediction, interaction prediction, real-time 3D detection, and real-time 2D detection. The winner of each challenge will receive $15,000, with second-place teams receiving $5,000 and third place $2,000.

“We hope this data will be useful for researchers working on behavior prediction in a wide variety of domains and industries,” said Anguelov. He added, “I know first-hand how incredibly important high-quality data is to tackling thorny AV research questions, and I’m incredibly proud of the Waymo research team’s efforts to continue evolving the dataset and making it applicable for an increasing number of research areas. We welcome community feedback on how to make our dataset even more impactful in future updates.”

Waymo recently began limited testing of its robotaxis in San Francisco. The Alphabet subsidiary is proceeding cautiously, using employee volunteers at this point to gather feedback about the technology’s performance. Waymo said that while it does plan to grow its presence there over time, “we don’t have any specific timelines to share about when (or where) we’ll be offering a public service.” San Francisco is also home to Cruise.

Waymo and Cruise were the leaders in the clubhouse according to the California’s Department of Motor Vehicles annual Disengagement Report that came out earlier this month. The report captured the number of times a human driver takes over control of an autonomous driving system being tested on the state’s public roads between December 2019 to November 2020.