Whether it be for package delivery, surveillance, data collection or entertainment, drones are expected to one day be a common sight in the skies. But to do so, drones will need to autonomously navigate unstructured environments.

Most of today’s drones use GPS, which works great when the drones are flying at high altitudes. But according to researchers, there could be problems if drones have to navigate autonomously at low altitudes through unstructured city streets with cars, cyclists or pedestrians.

This led researchers at the University of Zurich and the National Centre of Competence in Research NCCR Robotics to develop Drone Network (DroNet), a convolutional neural network (CNN) that can safely fly drones through a city at low altitudes by imitating how cars and bicycles navigate.

How DroNet was trained

To train DroNet on how to steer a drone, the researchers used a dataset collected by cars. And to train the system to avoid collisions, the team simply strapped a GoPro camera to a bicycle and rode around Zurich. Then by imitating cars and bicycles, the drone automatically learned the rules of the road.

DroNet produces two outputs from the data captured by the drone’s on-board camera:

• A steering angle to help the drone navigate while avoiding obstacles
• A collision probability that enables the drone to recognize dangerous situations – pedestrians or bicyclists – and react accordingly.

A paper detailing DroNet was recently published in the journal IEEE Robotics and Automation Letters. Check out DroNet in action in the video below:

“DroNet recognizes static and dynamic obstacles and can slow down to avoid crashing into them,” said Davide Scaramuzza, professor for robotics and perception at the University of Zurich. “With this algorithm we have taken a step forward towards integrating autonomously navigating drones into our everyday life.”

One thing the researchers find surprising is DroNet’s ability to generalize. Despite no indoor data being used to train it, DroNet learned to autonomously navigate indoor environments such as parking lots and offices.

DroNet was also designed to require little computational power compared to other CNNs. This allows real-time performance, even on a CPU, according to the team.

Deep Neural Networks for drones

DroNet isn’t the first time Scaramuzza used CNNs on drones. Two years ago, he used pictures taken with a camera on a hiker’s head to train a network. That network enabled a drone with minimal on-board sensing to autonomously navigate through a forest.

The idea was to study the problem of perceiving forest or mountain trails from a single monocular image captured by a robot traveling on the trail itself. Here’s video of that system in action:

As for DroNet, the researchers have shared their datasets and trained networks. And they admit that if DroNet is to play a significant role in the future of commercial drones, the algorithm needs to improve to enable faster, more agile flying.