Georgia Tech robots learn to ‘MacGyver’ simple tools

Thanks to new technology that enables them to create simple tools, robots may be on the verge of their own version of the Stone Age.

Using a novel capability to reason about shape, function, and attachment of unrelated parts, researchers have for the first time successfully trained an intelligent agent to create basic tools by combining objects.

The breakthrough comes from Georgia Tech’s Robot Autonomy and Interactive Learning (RAIL) research lab and is a significant step toward enabling intelligent agents to devise more advanced tools that could prove useful in hazardous – and potentially life-threatening – environments.

The concept may sound familiar. It’s called “MacGyvering,” based off the name of a 1980s — and recently rebooted — television series. In the series, the title character is known for his unconventional problem-solving ability using differing resources available to him.

For years, computer scientists and others have been working to provide robots with similar capabilities. In their new robot-MacGyvering work, RAIL lab researchers led by Associate Professor Sonia Chernova used as a starting point a robotics technique previously developed by former Georgia Tech Professor Mike Stilman.

In this latest work, a robot trained using the team’s novel approach is given a set of optional parts and told to make a specific tool. Much like its human counterparts, the robot first examines the shapes of each part and how one might be attached to another.

Using machine learning, the robot is trained to match form to function – which object shapes facilitate a particular outcome – from numerous examples of everyday objects. For example, by learning that the concavity of bowls enables them to hold liquids, it makes use of this knowledge when constructing a spoon. Similarly, the robots were taught how to attach objects together from examples of materials that could be pierced or grasped.

In the study, researchers successfully created hammers, spatulas, scoops, squeegees, and screwdrivers.

“The screwdriver was particularly interesting because the robot combined pliers and a coin,” said Lakshmi Nair, a Ph.D. student in the School of Interactive Computing and one of the researchers on the project. “It reasoned that the pliers were able to grasp something and said that the coin sort of matched the head of a screwdriver. Put them together, and it creates an effective tool.”

Currently, the robot is limited only to the shape and attachment. It cannot yet effectively reason about particular material properties, a crucial step in advancing to a real-world scenario.

“People reason that hammers are sturdy and strong, so you wouldn’t make a hammer out of foam blocks,” Nair said. “We want to reach that level of reasoning in our work, which is something we’re working on now.”

The inspiration for the work comes from the popular story of Apollo 13, the doomed seventh crewed flight of the Apollo space program. After an oxygen tank in the ship’s service module exploded two days into the mission, crew members were forced to make makeshift modifications to the carbon dioxide removal system.

Despite a dangerously tight window of time and extremely high tension among all aboard and at mission control, the rescue proved successful. Nair and collaborators hope this research will prove foundational to future robotics technology that could reason faster and without the burden of stress.

“They were able to make this filter, but the solution took a long time to come up with,” Nair said. “We want to make robots that can assist humans in these kinds of scenarios to take the pressure off of them to come up with innovative solutions and potentially save their lives.”

This work was presented at the 2019 Robotics: Science and Systems conference in a paper titled “Autonomous Tool Construction Using Part Shape and Attachment Prediction” (Lakshmi Nair, Nithin Shrivatsav, Zackory Erickson, Sonia Chernova). It is supported in part by grants from the National Science Foundation and the Office of Naval Research.

Editor’s Note: This article was republished from the Georgia Tech College of Computing.