A major limitation for aerial drones is the tradeoff between weight and battery capacity, which limits their range and usefulness for applications such as agriculture and infrastructure inspection. To address this challenge, a multidisciplinary team at National Tsing Hua University in Hsinchu, Taiwan, has developed an artificial intelligence processor that mimics the optical nerves of a fruit fly.

This AI chip enables unmanned aerial vehicles (UAVs) to automatically avoid obstacles while staying in an “ultra-power-saving mode,” said the researchers. The team was led by professors Tang Kea-tiong of the Department of Electrical Engineering and Lo Chung-chuan of the Department of Life Sciences at National Tsing Hua University (NTHU).

Most UAVs currently rely on the transmission and reflection of electromagnetic waves to detect and avoid obstacles, but this consumes a lot of power, they said. An alternative approach to avoiding obstacles is to use optical lenses to capture and analyze images, but the amount of information to be processed is too large to be done quickly, and this approach also consumes a lot of power.

Agile fruit fly provides inspiration

Intrigued by the fruit fly’s uncanny ability to avoid obstacles, Tang figured that it might be possible to replicate the optical nerve of this tiny insect and adapt it to AI applications.

The first task was to solve the problem of information overload. According to Tang, the image sensors currently used in cameras and mobile phones have millions of pixels, whereas the eye of a fruit fly has only about 800 pixels. When the fruit fly’s brain processes such visual signals as contour and contrast, it uses a detection mechanism that automatically filters out unimportant information. The fruit fly pay attention only to moving objects with which it could collide.

By imitating the fruit fly’s detection mechanism, the research team has developed an AI chip that makes it possible to use hand gestures and an image sensor to operate a drone.

First, the drone is taught to focus on what’s most important, and then it’s taught how to judge distance and the likelihood of a collision. Lo conducted a detailed investigation on how the fruit fly detects optical flow, for which he made extensive use of the maps of the fruit fly’s neural pathways produced by the Brain Research Center at NTHU.

“Optical flow is the relative trajectory left in the field of vision by nearby moving objects, and which is used by the brain to determine its distance and to avoid obstacles,” Lo explained.

Tang said that the AI chip developed by his research team represents a major breakthrough in the area of in-memory computing. Computers and mobile phones first move data from the memory to the central processing unit. Once it’s processed, the data is moved back to the memory for storage. Such a process can consume up to 90% of the energy and time of the AI deep-learning process.

By contrast, the NTHU team said its AI chip mimics the fruit fly neuronal synapses, allowing it to perform computations in the memory, which greatly improves efficiency.