Waymo continues to be the autonomous vehicle leader. (Credit: Waymo)
The California Department of Motor Vehicles (DMV) this week released the disengagement reports for companies testing autonomous vehicles on public roads in California. The data, captured between Nov. 31, 2017 through Dec. 1, 2018, found that 48 companies reported driving 2 million miles in autonomous mode on California roads and highways.
The California DMV defines a disengagement as a “deactivation of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle.”
Some have said this definition is too vague and has allowed companies to avoid reporting certain events. Others have said the data should be weighted based on the difficulty of the driving environment. Nevertheless, the data offers a barometer of the companies pushing the industry forward. And, not surprisingly, Waymo and GM Cruise continue to be way ahead of the pack.
According to the reported data, Waymo once again had the best-performing autonomous vehicles in California with one disengagement every 11,017 miles. That performance marks a 50 percent reduction in the rate and a 96 percent increase in the average miles traveled between disengagements compared to the 2017 numbers. In 2016, Waymo had one disengagement every 5,128 miles. Waymo also drove significantly more miles, up from 352,000 miles in 2017 to 1.2 million miles in 2018, which makes the performance even more impressive.
(Credit: Waymo)
“A lower rate of disengagements shows that our cars are getting better at recognizing and handling a wide variety of driving situations, including “edge cases” across the cities we’ve been testing in: those unusual situations that a human driver might see only once (or never) in a lifetime of driving,” Waymo wrote in a blog about its disengagement report. “As we continue to expand new territory or introduce new skills, disengagements will occur. Disengagements in these cases are actually a good thing because they are the equivalent to discovering and solving an issue with our car’s capability. As we continue to expand our test fleet, we actively seek places that present challenges to give our fleet the opportunity to learn.”
GM Cruise had the second-best performing autonomous vehicles in California last year with one disengagement every 5,204.9 miles. Cruise has 162 self-driving cars operating in California that drove a reported 447,621 miles. That was more than triple the mileage from 2017 for Cruise, which had 0.19 disengagements per 1,000 miles. That is also a significant improvement from Cruise’s 0.8 disengagements per 1,000 miles in 2017.
Cruise does most of its testing in San Francisco, which it claims is much more challenging than the Silicon Valley suburbs that are the main test bed for Waymo’s vehicles. Waymo did, however, expand its fleet into San Francisco last year. Kyle Vogt, Cruise’s co-founder and chief technology officer, said in 2017 that “every minute of testing in San Francisco is about as valuable as an hour of testing in the suburbs.”
The worst self-driving car performances belonged to Apple and Uber, which had a disengagement every 1.1 and 0.4 miles, respectively. This might explain why Apple reportedly laid off more than 200 employees from its self-driving car project. Not only was Apple late to the game, its performance was amateurish and shows its not ready to be a serious player in the space. Uber, of course, pulled its self-driving cars off the roads nationwide in late March 2018 after the fatal accident in Arizona.
Tesla said it “did not test any vehicles on public roads in California in autonomous mode or operate any autonomous vehicles, as defined by California law.”
Below is a breakdown of the 2018 disengagement reports. To read the full reports for each company, click here. It might be decades before autonomous vehicles are ubiquitous, as Waymo CEO John Krafcik has said. But that hasn’t negatively impacted investor interest as autonomous vehicles companies have already raised more than $1.6 billion in February 2019.
Autonomous Vehicle Disengagements 2018
| Company | Disengagements per 1000 miles (2018) | Miles per Disengagement (2018) | Miles Driven (2018) | Miles per disengagement (2017) |
|---|---|---|---|---|
| Waymo | 0.09 | 11,017 | 1,271,587 | 5,595.95 |
| GM Cruise | 0.19 | 5,204.9 | 447,621 | 1,254.06 |
| Zoox | 0.52 | 1,922.8 | 30,764 | 282.96 |
| Nuro | 0.97 | 1,028.3 | 24,680 | -- |
| Pony.ai | 0.98 | 1,022.3 | 16,356 | -- |
| Nissan | 4.75 | 210.5 | 5,473 | 208.36 |
| Baidu | 4.86 | 205.6 | 18,093 | 41.06 |
| AIMotive | 4.96 | 201.6 | 3,428 | -- |
| AutoX | 5.24 | 190.8 | 22,710 | -- |
| Roadstar.AI | 5.70 | 175.3 | 7,539 | -- |
| WeRide/JingChi | 5.71 | 173.5 | 15,440.80 | -- |
| Aurora | 10.01 | 99.9 | 32,858 | -- |
| Drive.ai | 11.91 | 83.9 | 4,616.69 | 43.59 |
| PlusAI | 18.40 | 54.4 | 10,816 | -- |
| Nullmax | 22.40 | 44.6 | 3,036 | -- |
| Phantom AI | 48.20 | 20.7 | 4,149 | -- |
| NVIDIA | 49.73 | 20.1 | 4,142 | 4.63 |
| SF Motors | 90.56 | 11 | 2,561 | -- |
| Telenav | 166.67 | 6.0 | 30 | 32 |
| BMW | 219.51 | 4.6 | 41 | -- |
| CarOne/Udelv | 260.27 | 3.8 | 219 | -- |
| Toyota | 393.70 | 2.5 | 381 | -- |
| Qualcomm | 416.63 | 2.4 | 240.02 | -- |
| Honda | 458.33 | 2.2 | 168 | -- |
| Mercedes Benz | 682.52 | 1.5 | 1,749.39 | 1.29 |
| SAIC | 829.61 | 1.2 | 634.03 | -- |
| Apple | 871.65 | 1.1 | 79,745 | -- |
| Uber | 2608.46 | 0.4 | 26,899 | -- |
At least with the updated numbers Apple is no longer last. Uber is now last.
But use Siri sometime and then Google assistant. Apple is just horrible at AI. It sucks.