OmniTact uses micro-cameras for multi-directional tactile sensing

A team of researchers at UC Berkeley recently developed a new multi-directional tactile sensor, called OmniTact, that overcomes some of the limitations of previously developed sensors. OmniTact, presented in a paper pre-published on arXiv, acts as an artificial fingertip that allows robots to sense the properties of objects it is holding or manipulating.

“Our lab recognized the need for a sensor for general robotic manipulation tasks with expanded capabilities than current tactile sensors,” said Frederik Ebert. “Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. For example, the GelSight sensor provides high resolution (i.e., 400×400 pixel) images but is large and flat, providing sensitivity on only one side, while the OptoForce sensor is curved, but only provides force readings at a single point.”

OmniTact, the sensor developed by Ebert and his colleagues, is an adaptation of GelSight, a tactile sensor created by researchers at MIT and UC Berkeley. GelSight can generate detailed 3-D maps of an object’s surface and detect some of its characteristics.

In contrast with GelSight, OmniTact is multi-directional, which means that all of its sides have sensing capabilities. In addition, it can provide high-resolution readings, is highly compact and has a curved shape. When integrated into a gripper or robotic hand, the sensor acts as a sensitive artificial ‘finger,” allowing the robot to manipulate and sense a wide range of objects varying in shape and sizes.

OmniTact was built by embedding multiple micro-cameras into an artificial skin made of silicone gel. The cameras detect multi-directional deformations of the gel-based skin, producing a rich signal that can then be analyzed by computer vision and image processing techniques to infer information about the objects that a robot is manipulating.

“Our team uses learning-based approaches to make use of the high-resolution feedback and avoid manual calibration procedures,” said Akhil Padmanabha. “OmniTact provides touch sensing on all sides of the sensor, thus allowing robots to have access to tactile feedback while making contact with objects in any direction.”

Ebert, Padmanabha and their colleagues evaluated their sensor’s performance on a fairly challenging task, where a robot had to insert an electrical connector into an outlet. They also tested its ability to infer the angle of contact of a finger as it pressed against a given object. In these initial evaluations, OmniTact achieved remarkable results, outperforming both a tactile sensor that is only sensitive on one side and another multi-directional sensor in the state estimation task.

Related: Machine learning helps researchers build low-cost tactile sensor

“Through our experiments with OmniTact, we discovered that multi-directional, high-resolution tactile sensing enables certain fine-grained manipulation tasks, such as the connector insertion task we presented,” said Stephen Tian, another researcher involved in the study. “We suspect that in the future multi-directional tactile sensing could be an essential element in general-purpose robotic manipulation in addition to applications such as robotic teleoperation in surgery, as well as in sea and space missions.”

The researchers built their sensor using micro-cameras that are typically used in endoscopes, casting the silicone gel directly onto the cameras. As a result, OmniTact is far more compact than previously developed GelSight sensors and could thus have a broader range of possible applications. In the future, it could pave the way for the creation of new robots with more sophisticated sensing capabilities.

“We plan to make OmniTact cheaper and more compact, allowing it to be used in a wider range of tasks. Our team additionally plans to conduct more robotic manipulation research that will inform future generations of tactile sensors,” Padmanabha said.