The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe

Inside Waymo’s safety benchmarks for robotaxis

By Brianna Wessling | October 4, 2022

waymo benchmarking performance

A conceptual illustration of the collision avoidance performance of the Waymo Driver. | Source: Waymo

While autonomous vehicles are being deployed in cities across the U.S., like Austin, Las Vegas, Miami, Phoenix and San Francisco, members of the public still have many questions about how safe the vehicles are and they compare to human drivers.

Currently, there aren’t any universally accepted approaches for evaluating the safety of autonomous driving systems, so Waymo, the self-driving unit of Google’s parent company Alphabet, released its Response Time paper and Collision Avoidance Benchmarking paper to provide more clarity on the safety benchmarks it holds for its vehicles. The papers outline how the Waymo Driver compares to average human drivers and better than average drivers.

To compare human drivers to the Waymo Driver, the company first needed to determine how well human drivers are able to avoid collisions. Typically, human response timing has been evaluated during controlled experiments in artificial environments. There, subjects are instructed to respond to well-defined stimulus signals, like a loud sound or a brake light onset. 

While this method can be effective, it misses a few key aspects of driving in the real world, according to Waymo. Human drivers don’t always behave based on well-defined instructions, and simulated scenarios don’t account for the urgency of an impending crash on the road. It can also be difficult for researchers to determine when to “start the clock” to gauge response time, especially in slowly unfolding scenarios. 

As a result, Waymo said these simulated models overestimate a human driver’s response time, predicting that humans will respond later, in fast-moving situations. It also underestimates a driver’s response time in slow-developing situations, where humans can struggle to respond with urgency. 

Waymo’s framework aims to be more accurate than typical models by using data from naturalistic driving studies. The framework it uses is based on two key ideas. The first is that humans decide to brake or steer to avoid a crash because something in their surroundings surprised them. The onset of this surprise is when Waymo starts the clock.

The second is that human responses will change depending on the evolving situation, so it can’t be represented by a single fixed response time that can be applied to all scenarios. 

Waymo used this framework and an evasive maneuver model to create an internal benchmark for collision avoidance behavior that exceeds typical human drivers. This reference model accounts for a human driver that is always paying attention, never impaired, and responds to every situation as quickly as possible. In Waymo’s words, it’s a driver that is Non-Impaired, with Eyes always On the conflict (NIEON).

To test how NIEON and the Waymo Driver perform against human drivers, Waymo compared the simulated performance of the Waymo driver to human drivers involved in a series of fatal crashes that happened between 2008 and 2017 in Chandler, Arizona. The company found that the Waymo Driver completely avoided or mitigated 100% of crashes, except for those where the driver was struck from behind. 

AV safety banchmarks

How the Waymo Driver compares to NIEON in collision avoidance tests. | Source: Waymo

Waymo then tested how NIEON would behave in these crashes, as NIEON would present a higher benchmark than average human drivers. They found that the NIEON model could prevent 62% of crashes entirely, and reduce serious injury risk by 84%. The Waymo Driver, however, still did better, preventing 75% of collisions and reducing serious injury risk by 93%. 

Waymo is delivering a keynote at RoboBusiness, which runs Oct. 19-20 in Santa Clara and is produced by WTWH Media, the parent company of The Robot Report. Allison Thackston, technical lead & manager at Waymo, will discuss on Oct. 19 from 9:30-10:15 AM the technical challenges of developing autonomous vehicles. She will also provide updates on Waymo’s ongoing deployments and work. Prior to Waymo, Thackston worked at Nuro, Toyota Research Institute and Oceaneering. She’s also a founding member of the ROS2 Technical Steering Committee.

About The Author

Brianna Wessling

Brianna Wessling is an Associate Editor, Robotics, WTWH Media. She joined WTWH Media in November 2021, after graduating from the University of Kansas with degrees in Journalism and English. She covers a wide range of robotics topics, but specializes in women in robotics, autonomous vehicles, and space robotics.

She can be reached at bwessling@wtwhmedia.com

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

Headshots of charlie kemp and jan zizka, developer of Brightpick's Giraffe, with podcast logo.
Reaching new heights: How Brightpick’s Giraffe can lift warehouse efficiency
A Lucid, Nuro, Uber vehicle.
Lucid, Nuro, Uber team up on global robotaxi fleet
Waymo picking up riders in San Francisco.
Waymo reaches 100M fully autonomous miles across all deployments
The Loomia Smart Skin Developer Kit can help roboticists test flexible tactile sensing, as shown here with robotic hands.
Loomia Smart Skin Developer Kit to help give humanoid robots a sense of touch

RBR50 Innovation Awards

“rr
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for Robotics Professionals.
The Robot Report Listing Database

Latest Episode of The Robot Report Podcast

Automated Warehouse Research Reports

Sponsored Content

  • How to Set Up a Planetary Gear Motion with SOLIDWORKS
  • Sager Electronics and its partners, logos shown here, will exhibit at the 2025 Robotics Summit & Expo. Sager Electronics to exhibit at the Robotics Summit & Expo
  • The Shift in Robotics: How Visual Perception is Separating Winners from the Pack
  • An AutoStore automated storage and retrieval grid. Webinar to provide automated storage and retrieval adoption advice
  • Smaller, tougher devices for evolving demands
The Robot Report
  • Automated Warehouse
  • RoboBusiness Event
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Contact Us

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe