The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
    • Security
  • Financial
    • Investments
    • Mergers & Acquisitions
    • Earnings
  • Resources
    • Careers
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50 Winners 2022
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
    • Leave a voicemail

Self-driving vehicles drove nearly 2.9M test miles in California

By Keith Shaw | February 27, 2020

waymo california

Waymo’s self-driving cars can now carry passengers in California.

Companies allowed to test autonomous vehicles in California with a safety driver reported driving nearly 2.9 million miles during the latest reporting period, an increase of more than 800,000 miles from the previous reporting cycle, the California Department of Motor Vehicles reported this week.

The state’s DMV released 60 annual reports submitted by companies testing vehicles in autonomous mode on California public roads during the reporting period – Dec. 1, 2018 through Nov. 30, 2019. Reports include the total number of disengagements, the circumstances or testing conditions, location, and total miles traveled in autonomous mode on public roads for each permit holder.

A disengagement can occur when a failure of the technology is detected, or when a safety driver decides to take control of the vehicle. The DMV said that reports provide insights on a company’s testing activities in California, “but are not intended to compare one company with another or reach broad conclusions on technological capabilities.”

Under California’s current regulations, companies are not required to report testing on private roads or test tracks, testing that occurs out of state, below SAE Level 3, or testing done in simulation. The DMV said 24 permit holders reported they did not test autonomous vehicles on California public roads. Two companies – Roadstar.AI and Xmotors.AI, failed to submit a disengagement report, and their permits were revoked on Feb. 26, 2020.

Currently 64 companies have valid permits to test autonomous vehicles with a safety driver on California public roads, and one company (Waymo) has a permit for driverless testing. With its driverless testing, Waymo reported no miles driven on public roads for this permit.

Similar to the 2018 reports, Waymo logged the most miles driven during the period – it recorded 1.45 million miles driven autonomously, with 110 disengagements. Next up was Cruise LLC, which drove 831,039 miles with 68 disengagements. Combined, those two companies represented 79% of all of the autonomous miles driven during the period.

Miles driven

The number of miles driven between the 2019 reports and 2018 reports indicate that many companies have ramped up their testing on California public roads. For example, Baidu USA ramped up its miles driven to 108,300.2 in 2019, compared with 18,093 miles in 2018. Waymo and Cruise also saw big increases in the miles driven between the two years, with Cruise logging 383,418 more miles in 2019 than in 2018, and Waymo increasing its miles driven by 182,550.

Companies reducing its miles driven were also shown in the report. This includes Apple, which reduced its miles driven in 2019 – it only logged 7,544 miles driven during the 2019 reporting period, compared with 79,745 miles in 2018. Aurora Innovation, Nissan, Phantom.AI, PlusAI, and WeRide also saw significantly fewer miles driven in 2019 compared with 2018.

This doesn’t necessarily mean that these companies are going away from the autonomous vehicle space, but rather they just show less testing on public roads. Many could be testing on private roads, testing in other states, or testing through simulation.

CompanyMiles driven (2019)Miles driven (2018)Increase/decrease
AImotive6,0563,4282,628
Apple7,54479,745-72,201
Aurora Innovation13,429.4932,858-19,428.51
AutoX Technologies32,05422,7109,344
Baidu USA108,300.218,09390,207.2
BMW of North America21.3641-19.64
Cruise LLC831,039.88447,621383,418.88
Drive.AI3,974.254,616.69-642.44
Lyft42,930.96042,930.96
Mercedes Benz R&D North America14,238.061,749.3912,488.67
Nissan North America2,277.45,473-3,195.6
Nullmax2,4303,036-606
Nuro68,761.9424,68044,081.94
NVIDIA7,2184,1423,076
Phantom AI1,1254,149-3,024
PlusAI1,88010,816-8,936
Pony.AI174,845.2916,356158,489.29
Qualcomm2,164.19240.021,924.17
SAIC Innovation2,229.66634.031,595.63
SF Motors3,492.82,561931.8
Telenav2230-8
Tesla12.2012.2
Toyota Research Institute1,8173811,436
Udelv706.55219487.55
Valeo North America99.56099.56
Waymo1,454,137.321,271,587182,550.32
WeRide5,91715,440.8-9,523.8
Zoox67,01530,76436,251
Total2,855,739.112,001,370,93854,368.18

Miles per disengagement

This is a tricky area to analyze – while it’s easy to take the number of miles driven and divide by disengagement to give a sense of how many miles a car has driven before a safety driver disengaged, this does not tell the whole story. Safety drivers and companies have different policies and procedures for when to disengage, as outlined by the company’s individual disengagement reports. Sometimes this can be weather related, sometimes it’s software, sometimes it’s another reckless or aggressive driver on the road. Subjective decisions by safety drivers make these numbers less solid in terms of deciding whether one company is better than another.

However, we are presenting these numbers to show whether an individual company has progressed in increasing the number of miles driven per disengagement between the 2018 and 2019 reporting periods. It is true that company policies could have changed to become more or less cautious during those periods – we would have no way of knowing this, however.

The “leader” in the clubhouse would be Baidu, which reported 18,050 miles driven per disengagement, followed by Waymo (13,219.43 miles/disengagement), Cruise (12,221.17 miles/disengagement), and AutoX (10,684.67). These companies are the only ones that were able to log more than 10,000 miles per disengagement.

Baidu’s leap into the top spot has raised some eyebrows. For example, in this blog post, the author questions Baidu’s 8,697% increase in its disengagement rate. Other sites, including The Verge, write that the reports are the only real glimpse that the public (and the media) have in terms of progress on autonomous vehicles.

Responding to the reports, Waymo put out a series of seven tweets commenting on the California disengagement reports and the process:

1/7 We appreciate what the California DMV was trying to do when creating this requirement, but the disengagement metric does not provide relevant insights into the capabilities of the Waymo Driver or distinguish its performance from others in the self-driving space.

— Waymo (@Waymo) February 26, 2020

“We appreciate what the California DMV was trying to do when creating this requirement, but the disengagement metric does not provide relevant insights intot he capabilities of the Waymo Driver or distinguish its performance from others in the self-driving space.”

“While much of the development, learning, and validation of the Waymo Driver comes through billions of miles driven within our simulation environments, our real-world driving experience is primarily outside of California, in markets like Detroit and Phoenix.”

“Most of our large-scale real-world driving, which is critical for full-system validation (including validating the realism of our simulator) comes from Phoenix. Indeed, our production releases are already serving thousands of Waymo riders in Phoenix.”

“This real-world production performance is largely unrelated to our California testing outputs. At this stage our real-world driving in California is predominately engineering development, and not production releases.”

“This is even more the case in 2019 and 2020, as we are now developing our 5th-generation Waymo Driver in Silicon Valley, SF and LA. We don’t think CA disengagement data should be used to compare performance, or judge readiness or competency.”

“We also expect our values to fluctuate depending on release timing, geography and other non-engineering reasons, which invalidates time-series comparisons.”

“Later this year, we’ll share more on the safety framework we’ve developed to ensure the safe performance of our self-driving technology as we continue to scale our fully driverless operations.”

CompanyDisengagements (2019)Miles/disengagement (2019)Miles/disengagement (2018)
AImotive26232.92201.60
Apple64117.881.10
Aurora Innovation14195.2499.90
AutoX Technologies310684.67190.8
Baidu USA618050.03205.60
BMW of North America82.674.60
Cruise LLC6812221.175,204.90
Drive.AI7552.9983.90
Lyft166725.750
Mercedes Benz R&D North America20546.931.5
Nissan North America4748.46210.5
Nullmax7034.7144.6
Nuro342022.411028.3
NVIDIA65511.0220.10
Phantom AI4326.1620.70
PlusAI294054.40
Pony.AI276475.751022.30
Qualcomm3758.492.4
SAIC Innovation4055.741.2
SF Motors14024.9511
Telenav37.336
Toyota Research Institute29470.622.5
Udelv4441.593.8
Valeo North America921.080
Waymo11013219.4311017.00
WeRide39151.72173.5
Zoox421595.61922.8

Then there’s Tesla. In the 2018 reports, the company announced they were testing on private roads, but in the 2019 reports they have driven a grand total of 12.2 miles with zero disengagements. This gives a divide-by-zero figure for their miles per disengagement, but it also indicates that it’s likely that their testing is continuing on private roads.

First-timers

First-time filers (those who received permits between December 2017 and November 2018) logged a total of 24,872.86 miles, the reports indicated.

CompanyDisengagementsMiles drivenMiles/disengagement
Ambarella523,16160.79
Apex.AI63448.257.12
BoxBot10938.50.35
DiDi Research812,279.421,534.93
Gatik AI242,553106.38
Intel Corp1651,2957.85
RideCell68.691.45
ThorDrive275,089188.48

Companies that have permits, but reported no autonomous testing on public roads, included:

  • AAA NCNU (did not have active permits at the submission deadline)
  • Aptiv Solutions
  • Bosch
  • Changan Automobile
  • Continental
  • CYNGN Inc
  • EasyMile
  • Faraday & Future Inc
  • Ford
  • Helm.AI Inc
  • Honda
  • Imagry Inc.
  • Mando America Corporation
  • Navya Inc.
  • Nio USA, Inc.
  • Renovo.auto
  • Samsung
  • Subaru
  • TORC Robotics Inc
  • TuSimple
  • Udacity, Inc.
  • Volkswagen Group of America
  • Voyage

What does this mean?

These reports give us some insights into where different companies are progressing in their testing of autonomous vehicles, at least on public roads in California. Some have argued that these disengagement reports are misleading, while others have argued that the reports mean that the world isn’t ready for self-driving cars.

For us, it’s a nice set of numbers that indicate that progress is being made by some companies, as well as to see the types of issues that are causing safety drivers to disengage (weather, other drivers, software issues). Apart from that, the rest is subjective.

About The Author

Keith Shaw

Keith Shaw is the Editor-in-chief of Robotics Business Review, a sister publication of The Robot Report. An award-winning journalist, he has worked as an editor for Network World, Computerworld and various newspapers across Massachusetts, New York, and Florida. He holds a degree in journalism from Syracuse University.

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

cruise robotaxis in San Francisco
Cruise hits milestone by charging for robotaxis rides in SF
csail simulation
MIT CSAIL releases open-source simulator for autonomous vehicles
The Cruise car in San Francisco
Nvidia patent helps autonomous cars detect emergency vehicles
Gatik disrupts class 8 short-haul market with new customers

2021 Robotics Handbook

The Robot Report Listing Database

Latest Robotics News

Robot Report Podcast

Anders Beck introduces the UR20; California bans autonomous tractors
See More >

Sponsored Content

  • Magnetic encoders support the stabilization control of a self-balancing two-wheeled robotic vehicle
  • How to best choose your AGV’s Wheel Drive provider
  • Meet Trey, the autonomous trailer (un)loading forklift
  • Kinova Robotics launches Link 6, the first Canadian industrial collaborative robot
  • Torque sensors help make human/robot collaborations safer for workers

RBR50 Innovation Awards

Leave us a voicemail

The Robot Report
  • Mobile Robot Guide
  • Collaborative Robotics Trends
  • Field Robotics Forum
  • Healthcare Robotics Engineering Forum
  • RoboBusiness Event
  • Robotics Business Review
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Advertising
  • Contact Us

Copyright © 2022 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
    • Security
  • Financial
    • Investments
    • Mergers & Acquisitions
    • Earnings
  • Resources
    • Careers
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50 Winners 2022
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
    • Leave a voicemail