Nvidia patent helps autonomous cars detect emergency vehicles

A Cruise autonomous vehicle operating in San Francisco. | Credit: Cruise

Nvidia filed a patent for a system that would help solve one of the biggest issues in autonomous driving: how self driving cars identify and react to emergency vehicles.

Autonomous vehicles can struggle when it comes to identifying emergency vehicles and responding to them appropriately. Just a few months ago, in April 2022, a Cruise vehicle in San Francisco inadvertently blocked a fire truck from responding to a fire. The truck, according to reporting from Wired, was trying to pass a double-parked garbage truck by using the opposing lane that a Cruise vehicle, without a safety driver, was in.

Typically, a human driver would simply back up so the emergency vehicle could get around it, but the Cruise vehicle stayed put for a crucial 25 seconds, before the garbage truck driver ran from their work to move the vehicle. According to a filing submitted to the California Public Utilities Commission, the incident resulted in a slowed response, meaning more property damage and personal injuries.

Nvidia’s patent filing, which was published by the US Patent and Trademark Office in May 2022, seeks to help self driving cars avoid situations like these.

The patent describes system involving microphones attached to an autonomous or semi-autonomous car to capture the sounds of nearby emergency response vehicles’ sirens. The microphones will work with a Deep Neural Network (DNN) to create audio signals that correspond to the sirens detected.

Those signals are then used to create a frequency spectrum that is then processed and analyzed by the DNN, which determines the location, direction of travel and the kind of emergency response vehicles. The vehicle can then make appropriate decisions based on where the emergency vehicle is and where it’s going.

While Nvidia’s patent has exciting implications for the world of autonomous vehicles, the company has not made any official product announcements, and the project seems to be at the patent application stage.

NVIDIA won a 2022 RBR50 Robotics Innovation Award from our sister publication Robotics Business Review. The company won for its Omniverse Replicator, a data generation engine that produces synthetic data for training deep neural networks based on physical simulations in photorealistic, physically-accurate virtual environments.