How California’s Self-Driving Cars Performed in 2017

The California Department of Motor Vehicles (DMV) released its annual report about disengagements of self-driving cars. The report analyzed 19 self-driving car companies that were permitted to test on Calif.’s public roads between December 2016 to November 2017.

The companies operated 230 autonomous vehicles during the 12-month period, which is more than double from 2016. But the major takeaway, unsurprisingly, is that Waymo is leading the pack. The next closest competitor is Cruise, which is owned by GM, while all the other companies are playing in the minor leagues. Waymo and Cruise accounted for nearly 76% of all autonomous miles driven in Calif. during the reporting period.

Waymo self-driving cars continue to lead industry

Waymo drove 352,545 miles autonomously in Calif. during the timeframe, which is down significantly from the 635,868 miles it autonomously drove in 2016. Waymo wrote in a Medium post that, overall, its autonomous miles doubled in 2017 to nearly 2 million. So the reduction in Calif. miles is due to Waymo focusing more of its attention elsewhere – mainly Phoenix, where it hopes to launch an autonomous ride-hailing service later in 2018.

Waymo also had a significant reduction in disengagements at 63. Six of these disengagements occurred on highways. That comes out to one disengagement every 5,596 miles. In 2016, Waymo had one disengagement every 5,128 miles. Waymo’s best month was November 2017 when it autonomously drove 30,516.7 miles and experienced just one disengagement.

Here’s the breakdown of what caused Waymo’s disengagements:

• 19 were caused by a Waymo vehicle making an “unwanted maneuver”
• 16 were caused a “perception discrepancy”
• 13 were caused by a “hardware discrepancy”
• 9 were caused by a “software discrepancy”
• 5 were caused by an “incorrect behavior prediction of other traffic participants”
• 1 was caused by “a recklessly behaving road user”

That means 98.4 percent of Waymo’s disengagements were due to technology failures. So, there’s still a ways to go. Waymo recently stopped publishing its monthly self-driving progress reports, and it said the DMV’s annual reports are not indicative of its overall testing program anymore.

Waymo’s performance proves its ready for the “thousands” of additional self-driving Chrysler Pacifica minivans it’s ordering. The company has been testing autonomous vehicles longer than any company and is the hands-down leader in the industry. As CEO John Krafcik said, “We’ve moved from research and development to operations and deployment.”

Those additional vehicles will help Waymo scale up, while everyone else plays catch-up.

GM Cruise self-driving cars a distant second

GM’s Cruise vehicles drove about a third less than Waymo, at 127,516 miles, but it still marked a 121,900-mile increase over 2016. Cruise reported 105 disengagements for an average of one disengagement every 1,214 miles.

The increased mileage has led to a number of accidents involving Cruise. For the reporting period, its autonomous vehicles were involved in 22 fender benders. That’s one crash for every 5,985 miles of testing. Not good. Cruise’s report to the DMV, unfortunately, doesn’t detail the accidents. Its fleet was involved in six accidents in September 2017 alone, but said none were its responsibility.

In Cruise’s report, you will notice that a lot of disengagements were caused by another “road user behaving poorly.” Interestingly, Cruise makes note of the difficulties that come with driving in San Francisco, which is where most of its mileage came from.

“We drive in San Francisco because it allows us to improve more quickly. Cities like San Francisco contain significantly more people, cars, and cyclists that our self-driving vehicles must be aware of at any given time. That makes San Francisco one of the hardest places to test a self-driving vehicle, and creates a rich environment for testing our object detection, prediction, and response functions. It also helps us validate our vehicles’ self-driving skills faster than testing in a suburban location alone. So, we drive here because by doing so we get better faster.”

Other self-driving cars playing catch-up

Nissan finished third, but it’s not much to brag about. Nissan drove 5,007 miles and had 24 disengagements, meaning its vehicles disengaged every 208 miles. Zoox was fourth with a disengagement every 160 miles. Mercedes had the worst performance of the group. It experienced a disengagement almost every mile. Bosch didn’t do much better with a disengagement every 2-3 miles.

BMW, Ford, Honda, NIO, Tesla, Volkswagen and Wheego, while permitted to do so, did not conduct autonomous driving tests on California’s roads during the time period. Some of these companies, however, are testing on closed-courses in California and testing in other countries and states. The DMV report does not cover those tests.

Uber said Wednesday it will not be required to file a disengagement report until January 2019. After being forced to pull its cars off the road, Uber applied for and received a permit last spring to test in Calif.

Although federal regulators have yet to determine how safe autonomous vehicles must be before they are introduced in large numbers, disengagements in one way to measure reliability. And it looks like Waymo’s self-driving vehicles are hands down the most reliable on Calif. roads.