The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe

How shadows can help robots understand human touch

By David Nutt | February 9, 2021

shadow

By placing a camera inside a robot, researchers can infer how the person is touching it and what the person’s intent is just by looking at the shadow images. | Credit: Cornell University

Cornell University researchers created a low-cost method for soft robots to detect a range of physical interactions, from pats to punches to hugs, without relying on touch at all. Instead, a USB camera located inside the robot captures the shadow movements of hand gestures on the robot’s skin and classifies them with machine-learning software.

The new ShadowSense technology is the latest project from Cornell’s Human-Robot Collaboration and Companionship Lab. The group published a paper on the research called “ShadowSense: Detecting Human Touch in a Social Robot Using Shadow Image Classification.”

“Touch is such an important mode of communication for most organisms, but it has been virtually absent from human-robot interaction,” said Guy Hoffman, associate professor at Cornell’s Sibley School of Mechanical and Aerospace Engineering and the paper’s senior author. “One of the reasons is that full-body touch used to require a massive number of sensors, and was therefore not practical to implement. This research offers a low-cost alternative.”

The technology originated as part of a collaboration to develop inflatable robots that could guide people to safety during emergency evacuations. Such a robot would need to be able to communicate with humans in extreme conditions and environments. Imagine a robot physically leading someone down a noisy, smoke-filled corridor by detecting the pressure of the person’s hand.

Rather than installing a large number of contact sensors – which would add weight and complex wiring to the robot, and would be difficult to embed in a deforming skin – the team took a counterintuitive approach. In order to gauge touch, they looked to sight.

“By placing a camera inside the robot, we can infer how the person is touching it and what the person’s intent is just by looking at the shadow images,” said Yuhan Hu, the paper’s lead author. “We think there is interesting potential there, because there are lots of social robots that are not able to detect touch gestures.”

Prototype robot

The prototype robot consists of a soft inflatable bladder of nylon skin stretched around a cylindrical skeleton, roughly four feet in height, that is mounted on a mobile base. Under the robot’s skin is a USB camera, which connects to a laptop. The researchers developed a neural network-based algorithm that uses previously recorded training data to distinguish between six touch gestures – touching with a palm, punching, touching with two hands, hugging, pointing and not touching at all – with an accuracy of 87.5 to 96%, depending on the lighting.

The robot can be programmed to respond to certain touches and gestures, such as rolling away or issuing a message through a loudspeaker. And the robot’s skin has the potential to be turned into an interactive screen.

By collecting enough data, a robot could be trained to recognize an even wider vocabulary of interactions, custom-tailored to fit the robot’s task, Hu said. The robot doesn’t even have to be a robot. ShadowSense technology can be incorporated into other materials, such as balloons, turning them into touch-sensitive devices.

“While the technology has certain limitations, for example requiring a line of sight from the camera to the robot’s skin, these constraints could actually spark a new approach to social robot design that would support a visual touch sensor like the one we proposed,” Hoffman said. “In the future, we would like to experiment with using optical devices such as lenses and mirrors to enable additional form factors.”

Increasing privacy

In addition to providing a simple solution to a complicated technical challenge, and making robots more user-friendly to boot, ShadowSense offers a comfort that is increasingly rare in these high-tech times: privacy.

“If the robot can only see you in the form of your shadow, it can detect what you’re doing without taking high fidelity images of your appearance,” Hu said. “That gives you a physical filter and protection, and provides psychological comfort.”

The ability to physically interact and understand a person’s movements and moods could ultimately be just as important to the person as it is to the robot.

“Touch interaction is a very important channel in terms of human-human interaction. It is an intimate modality of communication,” Hu said. “And that’s not easily replaceable.”

Editor’s Note: This article was republished from Cornell University.

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

Two cobot arms putting a gear on a bike tire.
TRI: pretrained large behavior models accelerate robot learning
reachy mini sitting on a desktop.
Hugging Face launches Reachy Mini robot as embodied AI platform
Mobile robots lifting racks in a warehouse.
Interact Analysis slashes its mobile robot outlook amid tariff uncertainty
The Surgical Robot Transformer-Hierarchy performing a gallbladder surgery.
Johns Hopkins teaches robot to perform a gallbladder removal on a realistic patient

RBR50 Innovation Awards

“rr
EXPAND YOUR KNOWLEDGE AND STAY CONNECTED
Get the latest info on technologies, tools and strategies for Robotics Professionals.
The Robot Report Listing Database

Latest Episode of The Robot Report Podcast

Automated Warehouse Research Reports

Sponsored Content

  • How to Set Up a Planetary Gear Motion with SOLIDWORKS
  • Sager Electronics and its partners, logos shown here, will exhibit at the 2025 Robotics Summit & Expo. Sager Electronics to exhibit at the Robotics Summit & Expo
  • The Shift in Robotics: How Visual Perception is Separating Winners from the Pack
  • An AutoStore automated storage and retrieval grid. Webinar to provide automated storage and retrieval adoption advice
  • Smaller, tougher devices for evolving demands
The Robot Report
  • Automated Warehouse
  • RoboBusiness Event
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Contact Us

Copyright © 2025 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Humanoids
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Business
    • Financial
      • Investments
      • Mergers & Acquisitions
      • Earnings
    • Markets
      • Agriculture
      • Healthcare
      • Logistics
      • Manufacturing
      • Mining
      • Security
    • RBR50
      • RBR50 Winners 2025
      • RBR50 Winners 2024
      • RBR50 Winners 2023
      • RBR50 Winners 2022
      • RBR50 Winners 2021
  • Resources
    • Automated Warehouse Research Reports
    • Digital Issues
    • eBooks
    • Publications
      • Automated Warehouse
      • Collaborative Robotics Trends
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
  • Advertise
  • Subscribe