The Robot Report

  • Research
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • Grippers / End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors / Sensing Systems
    • Soft Robotics
    • Software / Simulation
  • Development
    • A.I. / Cognition
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Defense / Security
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
  • Investments
  • Resources
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50
    • Search Robotics Database
    • Videos
    • Webinars
  • Events
    • RoboBusiness Direct
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
  • Podcast

Rubik’s Cube gets single-handed robotic solution with OpenAI training

By Eugene Demaitre | October 15, 2019

Most robotic grippers do not closely resemble human hands because they are designed for a limited range of functions or high precision and repeatability. However, human hands can be very dexterous and perform feats difficult for robots. The key to robotic manipulation is not the hardware but the software, according OpenAI. The company posted to its blog today about how it trained a robot hand to solve a Rubik’s Cube.

San Francisco-based OpenAI has been working on artificial general intelligence, in which robots learn to solve problems independently rather than be programmed with specific solutions. In July, Microsoft Corp. said it was investing $1 billion in OpenAI and partnering with it to develop AI on the Azure platform.

OpenAI’s blog post refers to a research paper its team wrote explaining how models trained in simulation could “solve a manipulation problem of unprecedented complexity on a real robot.”

The company has been working since May 2017 to train a robot hand to solve a Rubik’s Cube. While it was able to do so in simulation by July 2017, the physical robot achieved that capability only in July 2019.

The goal is to help train robots to eventually be general-purpose household assistants. Mobile manipulators for have also received interest for e-commerce order fulfillment, packing, manufacturing, and other tasks.

Applying machine learning to complex manipulation

“Solving a Rubik’s Cube one-handed is a challenging task even for humans, and it takes children several years to gain the dexterity required to master it,” said OpenAI. “Our robot still hasn’t perfected its technique, though, as it solves the Rubik’s Cube 60% of the time (and only 20% of the time for a maximally difficult scramble).”

The goal wasn’t just to solve a Rubik’s Cube, which other robots can do faster, but to be able to manipulate it without having data on all possible orientations and combinations first.

To get to that point, OpenAI kept the hardware it has been using for the past 15 years — a Shadow Dextrous E Series Hand — with a PhaseSpace motion-capture system for coordinating the five fingertips. The company also kept its 3 RGB Basler camera for visual pose estimation. It made only minor modifications for grip and and robustness to the Dactyl system.

The researchers did modify the Rubik’s Cube for its testing to include built-in sensors and a Bluetooth module. This enabled the cube to report its state and helped with the manipulation and testing.

While Dactyl’s hardware remained mostly the same, OpenAI’s latest research was different because of the techniques it used with two neural networks. It included the custom robot platform and automatic domain randomization (ADR). Normal randomization was not enough to train AI and robots to apply generalized lessons.

“The biggest challenge we faced was to create environments in simulation diverse enough to capture the physics of the real world,” OpenAI wrote. “Factors like friction, elasticity and dynamics are incredibly difficult to measure and model for objects as complex as Rubik’s Cubes or robotic hands, and we found that domain randomization alone is not enough.”

Rubik's Cube simulation

Automatic domain randomization enables a robot to be trained to solve a Rubik’s Cube in simulation. Source: OpenAI

ADR generated simulations of increasing complexity, and the control policy learned to solve them using a recurrent neural network and reinforcement learning. The convolutional neural network for pose prediction was trained on the same data but separately from the control policy, said OpenAI.

“Control policies and vision-state estimators trained with ADR exhibit vastly improved sim2real [simulation-to-reality] transfer,” stated OpenAI. “For control policies, memory-augmented models trained on an ADR-generated distribution of environments show clear signs of emergent meta-learning at test time.”

Transfer to real-world Rubik's Cube

By using ADR, it was easier to transfer lessons from simulation to a real-world Rubik’s Cube. Source: OpenAI

By “meta-learning,” OpenAI meant that the algorithm — and, by extension, robots — should be able to learn without prior knowledge and react accordingly to unforeseen factors in the environment. MIT and other research institutions are also working on the problem.

Overcoming random obstacles to Rubik’s Cube solution

As a neural network got better at solving the Rubik’s Cube, the amount of domain randomization is automatically increased, forcing the network to generalize its lessons. Random factors included the size and mass of the cube, the amount of friction, and the visible parts of the hand itself.

In addition to setting the challenges of manipulating and solving the Rubik’s Cube, the researchers added a rubber glove, a blanket, and a stuffed giraffe as environmental obstacles.

After repeated simulations and randomizations, the robot exceeded performance thresholds for both manipulating the block and solving the puzzle.

“We find that our system trained with ADR is surprisingly robust to perturbations, even though we never trained with them,” said OpenAI. “The robot can successfully perform most flips and face rotations under all tested perturbations, though not at peak performance.”

OpenAI found that visually representing how the neural networks solve problems helped associate semantic behaviors with the data gathered during simulations. This provided insight into the steps the algorithm took to move and solve the Rubik’s Cube.

While a Rubik’s Cube might seem a long way from figuring out how to open a refrigerator and fetch a beverage, developing human-level dexterity is an important step toward service robots that can observe, decide, and react to a wide variety of circumstances, said OpenAI.

About The Author

Eugene Demaitre

Eugene Demaitre was senior editor of The Robot Report from 2019-2020. Prior to working at WTWH Media, he was an editor at BNA (now part of Bloomberg), Computerworld, TechTarget, and Robotics Business Review. Demaitre has participated in robotics webcasts and conferences worldwide. He has a master's from the George Washington University and lives in the Boston area.

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

Leadec Diversified Automation
Leadec acquires Diversified Automation to expand robotics play
Oxbotica autonomous vehicle software
Oxbotica raises $47M to scale autonomous vehicle software
Isaac Gym is NVIDIA's reinforcement learning accelerator for robotics
Isaac Gym is NVIDIA’s reinforcement learning accelerator for robotics
Digital twins, AR/VR bring simulation benefits to industrial automation users, says Siemens
Digital twins, AR/VR bring simulation benefits to industrial automation users, says Siemens

Robotics Year in Review

The Robot Report Listing Database

Latest Robotics News

Robot Report Podcast

Teradyne’s acquisition strategy & the future of cobot

The Robot Report Podcast · Teradyne's acquisition strategy & the future of cobots

Sponsored Content

  • Doosan Robotics: Driving Innovation and Growth in Cobots
  • FORT Robotics Podcast: FORT Robotics on how to keep humans safe and in control of robots
  • Pallet Detection Systems Help Automated Forklifts Modernize Warehouse Operations
  • IES Servo Control Gripper
  • How to cut the cost of manufacturing

Tweets by RoboticTips

The Robot Report
  • Collaborative Robotics Trends
  • Field Robotics Forum
  • Healthcare Robotics Engineering Forum
  • RoboBusiness Event
  • Robotics Business Review
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Advertising
  • Contact Us

Copyright © 2021 WTWH Media, LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media. Site Map | Privacy Policy | RSS

Search The Robot Report

  • Research
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • Grippers / End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors / Sensing Systems
    • Soft Robotics
    • Software / Simulation
  • Development
    • A.I. / Cognition
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Defense / Security
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
  • Investments
  • Resources
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50
    • Search Robotics Database
    • Videos
    • Webinars
  • Events
    • RoboBusiness Direct
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
  • Podcast