Cruise, Waymo lead way in Calif. autonomous vehicle testing

California’s Department of Motor Vehicles (DMV) yesterday released its annual Disengagement Report, which breaks down the number of times a human driver takes over control of an autonomous driving system being tested on the state’s public roads. The DMV released reports for 29 companies for the period from December 2019 to November 2020.

Waymo (29,944.69) and Cruise (28,520.34) are the leaders in the clubhouse when it comes to the number of miles driven between disengagements. China’s AutoX (20,367) is a distant third. This is the sixth straight year in which Waymo increased its number of miles driven between disengagements.

Cruise, which is owned by General Motors, drove more than 770,000 miles last year to overtake Waymo, which has historically owned the top spot. Waymo drove just under 629,000 miles in 2020 after logging more than 1.4 million miles in California in 2019. Both companies drove more miles in 2019, but many self-driving car operations were shut down during part of 2020 due to the COVID-19 pandemic.

Zoox, which was acquired by Amazon in 2020 for $1.2 billion, drove 102,521 miles and had a disengagement every 1,627.32 miles. Aurora, which acquired Uber ATG in December 2020, drove just 12,208 miles in California and had a disengagement every 329.93 miles.

The chart below compiles data from each company’s performance in 2020 and their respective miles per disengagement in 2019. It is ordered highest to lowest based on the number of miles driven between disengagements in 2020. We have more information about the 2019 data here.

2020 Calif. Self-Driving Car Disengagements

Company	2020 Miles Driven	2020 Disengagements	2020 Miles/Disengagement	2019 Miles/Disengagement
Waymo	628,839	21	29,944.69	13219.43
Cruise	770,049	27	28,520.34	12221.17
AutoX	40,734	2	20,367.00	10684.67
Pony.ai	225,496	21	10,737.90	6475.75
Argo AI	21,037	2	10,518.59	N/A
WeRide	13,014	2	6,507.00	151.72
DiDi	10,401	2	5,200.75	N/A
Nuro	55,370	11	5,033.62	2022.41
DeepRoute	10,018	3	3,339.33	N/A
Zoox	102,521	63	1,627.32	1595.6
QCraft	7,582	16	473.88	N/A
Aurora	12,208	37	329.93	95.24
Lyft	32,731	123	266.11	25.75
Gatik	2,352	11	213.82	N/A
Apple	18,805	130	144.66	117.88
Nissan	395	4	98.63	48.46
BMW	122	3	40.67	2.67
Aimotive	2,987	113	26.43	232.92
Mercedes	29,984	1,167	25.69	6.93
NVIDIA	3,033	125	24.26	11.02
Qualcomm	1,727	90	19.19	58.49
SF Motors	875	61	14.34	24.95
Atlas	47	10	4.74	N/A
EasyMile	424	128	3.31	N/A
Toyota	2,875	1,215	2.37	0.62
Telenav	4	2	2	7.33
Udelv	66	49	1.35	1.59
Ridecell	148	189	0.78	N/A
Valeo	49	99	0.49	1.08

Argo AI, Atlas Robotics, DeepRoute, and QCraft reported numbers for a full year for the first time. Interestingly, 34 companies licensed to test self-driving cars on California’s public roads reported no testing took place. Tesla is one of those companies. Under California’s rules, there is no need to report data on systems below SAE Level 3. Tesla’s “Full Self-Driving” system is considered a SAE Level 2 driver assist system and not autonomous driving.

Seven companies (Bosch, Changan, Faraday Future, SAIC, Samsung, Torc Robotics, TuSimple) didn’t even renew their license. The license for Kaizr was suspended as of Feb. 9, 2021 as the company failed to submit a disengagement report despite conducting on-road testing.

Most autonomous vehicle companies, including Cruise and Waymo, have said disengagement reports are misleading for a variety of reasons. One major reason is the numbers are self-reported, so accuracy is always a question. But without a standard way to evaluate a self-driving car’s performance, this is the best information available. And for the first time, the California DMV shared who initiated the disengagement – the safety driver or the system – and where the disengagement occurred.

“For all of 2020, we averaged only one reportable disengagement per 28,520 miles,” Cruise wrote in a blog on Tuesday. “For the second half of 2020, we improved to more than 60,000 miles between reportable disengagement. And in the final three months of the year, we had zero reportable disengagements, a 10x improvement over the same period a year before.”

Waymo wrote on Twitter that while it appreciates what the DMV is trying to do with the report, “the metrics provide limited value in assessing the capabilities of the Waymo Driver, or in distinguishing its performance from other A/V companies.

“Further, our testing program spans different vehicles, software platforms, testing environments and objectives, so the disengagement rate in any given year simply can’t be predictive of real world performance.”

AutoX, which recently launched a robotaxi service for the public in China, drove only 40,734 miles. It experienced a disengagement every 20,637 miles, which is nearly double its performance last year.