NEW YORK — Developing a humanlike sense of touch has been challenging for roboticists. Researchers at Columbia Engineering here recently announced that they have created a robot finger that can localize touch with a high degree of precision over a large, multicurved surface.
“There has long been a gap between stand-alone tactile sensors and fully integrated tactile fingers — tactile sensing is still far from ubiquitous in robotic manipulation,” stated Matei Ciocarlie, an associate professor in the departments of mechanical engineering and computer science, who led this work in collaboration with Electrical Engineering Professor Ioannis (John) Kymissis. “In this paper, we have demonstrated a multicurved robotic finger with accurate touch localization and normal force detection over complex 3D surfaces.”
Columbia Engineering, also known as The Fu Foundation School of Engineering and Applied Science, has more than 220 faculty and is part of Columbia University.
Finding an easier way to build a robot finger
Current methods for building touch sensors have proven difficult to integrate into robot fingers due to because of multiple challenges, including difficulty in covering multicurved surfaces, a high wire count, or difficulty fitting into small fingertips, thus preventing use in dexterous hands. The Columbia Engineering team took a new approach: It used overlapping signals from light emitters and receivers embedded in a transparent waveguide layer that covers the functional areas of the finger.
By measuring light transport between every emitter and receiver, the researchers showed that they can obtain a very rich signal data set that changes in response to deformation of the robot finger due to touch. They then demonstrated that purely data-driven deep learning methods can extract useful information from the data, including contact location and applied normal force, without the need for analytical models.
The final result is a fully integrated, sensorized robot finger, with a low wire count, built using accessible manufacturing methods and designed for easy integration into dexterous hands.
Touching with light
The study, led by students Pedro Piacenza and Keith Behrman, was published online in IEEE/ASME Transactions on Mechatronics. It demonstrates the two aspects of the underlying technology that combine to enable the new results.
Firstly, in this project, the researchers use light to sense touch. Under the “skin,” the robot finger has a layer made of transparent silicone, into which they shined light from more than 30 LEDs. The finger also has more than 30 photodiodes that measure how the light bounces around. Whenever the finger touches something, its skin deforms, so light shifts around in the transparent layer underneath.
Measuring how much light goes from every LED to every diode, the researchers ended up with close to 1,000 signals that each contain some information about the contact that was made. Since light can also bounce around in a curved space, these signals can cover a complex 3D shape such as a fingertip.
“The human finger provides incredibly rich contact information — more than 400 tiny touch sensors in every square centimeter of skin!” said Ciocarlie. “That was the model that pushed us to try and get as much data as possible from our finger. It was critical to be sure all contacts on all sides of the finger were covered–we essentially built a tactile robot finger with no blind spots.”
Applying deep learning to robot finger
Secondly, the team designed this data to be processed by machine learning algorithms. Because there are so many signals, all of them partially overlapping with each other, the data is too complex to be interpreted by humans. Fortunately, current machine learning techniques can learn to extract the information that researchers care about: where the finger is being touched, what it is touching the finger, how much force is being applied, etc.
“Our results show that a deep neural network can extract this information with very high accuracy,” said Kymissis, who is also a member of the Data Science Institute. “Our device is truly a tactile finger designed from the very beginning to be used in conjunction with AI algorithms.”
In addition, the team built the finger so it, and others, can be put onto robotic hands. Integrating the system onto a hand is easy: thanks to this new technology, the finger collects almost 1,000 signals, but only needs a 14-wire cable connecting it to the hand, and it needs no complex off-board electronics.
The researchers already have two dexterous hands capable of grasping and manipulating objects in their lab. One is being outfitted with three fingers, and the other one four. In the next months, the team will be using these hands to try and demonstrate dexterous manipulation abilities, based on tactile and proprioceptive data.
“Dexterous robotic manipulation is needed now in fields such as manufacturing and logistics, and is one of the technologies that, in the longer term, are needed to enable personal robotic assistance in other areas, such as healthcare or service domains,” added Ciocarlie.