The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
    • Security
  • Financial
    • Investments
    • Mergers & Acquisitions
    • Earnings
  • Resources
    • Careers
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50 Winners 2022
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
    • Leave a voicemail

Toyota Research Institute teaches mobile manipulator with VR, simulation

By Eugene Demaitre | October 7, 2019


Robust mobile manipulation, which is starting to appear in supply chain operations, is a step toward service robots with more generalized capabilities. Toyota Research Institute last week posted a video (above) showing the state of its research on household robots.

Aging populations in Europe, East Asia, and North America pose challenges, according to a United Nations study cited by Toyota Research Institute (TRI). One of the goals of service robotics is to enable people to “age in place,” or remain in their homes with as much autonomy as possible and without scarce human aid.

Unlike other technology enterprises and automakers working on self-driving cars, Toyota has positioned itself as a “mobility company” as it looks to diversify its offerings beyond vehicles and into assisting households. As part of that effort, the company set an internal challenge of teaching robots to perform a variety of tasks in actual homes.

“Teaching robots to perform tasks is a key element of advancing fleet learning for enabling useful robot assistants in homes,” Jeremy Ma, senior manager of robotics at TRI, told The Robot Report. “Using an immersive telepresence system makes teaching and performing these tasks much more efficient, and we hope our progress can benefit others in the robotics community.”

Perception, navigation, and mobile manipulation in the household

Rather than map each home, as iRobot has done with recent floor-cleaning robots, TRI wanted its autonomous mobile robot to move around on its own as humans do.

The prototype robot chooses a path based on what it has learned, explained Dan Helmick, software engineer and senior robotics manager at TRI. He is also co-author of “A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes,” a study published by Cornell University. The robot can stop if it encounters something unexpected, Helmick said.

TRI’s mobile manipulator uses a combination of off-the-shelf and custom hardware. It includes four Intel RealSense cameras and two 5-megapixel Basler cameras for a wide field of view. The robot also has an ATI mini-45 force/torque sensor and runs on an 18-core Intel i9 CPU and an Nvidia TitanV graphics processing unit running Linux.

The vaguely humanoid robot uses industrial arms rather than collaborative arms, or cobots, because it is designed for ease of learning, not yet to operate alongside people, said TRI.

Joints in its Kinova Jaco2 arms and body provide 31 redundant degrees of freedom so that it can turn and reach items in, say, a kitchen cabinet, as needed. They include maxon motors and planetary gearheads. The robot also uses Sake Robotics grippers with under-actuated fingers, TRI said.

Toyota Research Institute teaches mobile manipulators

A major challenge for making robots smarter has been the massive amounts of data that they need to be trained on. TRI claimed that its methods, which include using simulations and virtual reality (VR) to train a robot to conduct certain tasks, require less data.

“More is not necessarily better,” said Helmick.

By teaching a robot about parameters rather than specific items, it can learn with just a few examples of what parts of an object are important for grasping, said Ma. The mobile manipulator can also learn how to move on uneven surfaces, interpret voice commands, and conduct complex tasks, he said.

After a single demonstration by a human using VR, the robot can autonomously perform multi-step tasks, said TRI. Toyota’s mobile manipulation system also includes dense visual representations of a scene with embedded objects, and a task graph of taught behaviors.

With simulations, a mobile manipulator can be trained to handle complex tasks such as loading a dishwasher, said Russ Tedrake, vice president of robotics research at TRI. Rather than train a robot on each item, it can learn how to reliably handle multiple objects.

TRI mobile manipulator software

TRI’s software architecture enables autonomous execution of taught tasks by processing visual and audio data, building a world model, mapping visual inputs to taught behaviors, and executing sequences of behaviors. Source: Toyota Research Institute

Sharing lessons with fleet learning

For robots to truly spread in people’s daily lives, they need to become useful for a wide variety of applications. Cloud computing, machine vision, and deep learning will lead to a flowering of capabilities similar to the “Cambrian Explosion” wrote TRI CEO Gill Pratt.

In short, the idea is that if one robot learns, that lesson can be shared among many robots rather than programming each one for a specific task or environment.

According to TRI, its mobile manipulator “can successfully perform a relatively complex human-level task about 85% of the time.” Each of the three tasks consisted of up to 45 behaviors, such as moving in a kitchen, opening and closing a refrigerator door, grasping a cup or container, and putting it on a table. Recoverable failures for individual behaviors — things the robot needed to do again — had a 99.6% success rate.


The Robot Report is launching the Healthcare Robotics Engineering Forum, which will be on Dec. 9-10 in Santa Clara, Calif. The conference and expo will focus on improving the design, development, and manufacture of next-generation healthcare robots. Learn more about the Healthcare Robotics Engineering Forum, and registration is now open.


Toyota Research Institute aims to help all service robots

Toyota Research Institute noted that its approach could be useful for teaching mobile manipulators in manufacturing, pick-and-place operations, packing, and logistics, but much work remains to be done for generalizing tasks. VR, teleoperation, and fleet learning could make robots more flexible and easier to instruct. TRI said it is improving key behaviors for all robots.

Rather than replacing human workers, TRI said it is working on robots to “amplify, assist, and empower human performance.” Toyota AI Ventures, which launched its second $100 million fund in May, has been investing in startups such as Freedom Robotics with similar goals.

In August, Toyota Motor Corp. announced a partnership with Preferred Networks Inc. to develop the Human Support Robot platform.

“We believe teaching a robot tasks is a promising first step to achieving our broader vision of fleet learning, specifically for assisting and empowering people in their home,” stated Toyota Research Institute.

About The Author

Eugene Demaitre

Eugene Demaitre was senior editor of The Robot Report from 2019-2020. Prior to working at WTWH Media, he was an editor at BNA (now part of Bloomberg), Computerworld, TechTarget, and Robotics Business Review. Demaitre has participated in robotics webcasts and conferences worldwide. He has a master's from the George Washington University and lives in the Boston area.

Comments

  1. Jérôme Bédard says

    October 7, 2019 at 3:20 pm

    Hello Eugene! Wonderful article!

    Just a Typo: it’s Kinova Jaco2 arm! (not a K).

    Thanks!

    Reply
    • Steve Crowe says

      October 7, 2019 at 3:27 pm

      Jerome, sorry about that. All set.

      Reply

Tell Us What You Think! Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Related Articles Read More >

myoshirt
ETH Zurich develops wearable muscles
csail simulation
MIT CSAIL releases open-source simulator for autonomous vehicles
A3 robots
Robot sales hit record high in first quarter of 2022
tiny robot on penny
Researchers create walking robot half a millimeter wide

2021 Robotics Handbook

The Robot Report Listing Database

Latest Robotics News

Robot Report Podcast

Anders Beck introduces the UR20; California bans autonomous tractors
See More >

Sponsored Content

  • Magnetic encoders support the stabilization control of a self-balancing two-wheeled robotic vehicle
  • How to best choose your AGV’s Wheel Drive provider
  • Meet Trey, the autonomous trailer (un)loading forklift
  • Kinova Robotics launches Link 6, the first Canadian industrial collaborative robot
  • Torque sensors help make human/robot collaborations safer for workers

RBR50 Innovation Awards

Leave us a voicemail

The Robot Report
  • Mobile Robot Guide
  • Collaborative Robotics Trends
  • Field Robotics Forum
  • Healthcare Robotics Engineering Forum
  • RoboBusiness Event
  • Robotics Business Review
  • Robotics Summit & Expo
  • About The Robot Report
  • Subscribe
  • Advertising
  • Contact Us

Copyright © 2022 WTWH Media LLC. All Rights Reserved. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of WTWH Media
Privacy Policy | Advertising | About Us

Search The Robot Report

  • Home
  • News
  • Technologies
    • Batteries / Power Supplies
    • Cameras / Imaging / Vision
    • Controllers
    • End Effectors
    • Microprocessors / SoCs
    • Motion Control
    • Sensors
    • Soft Robotics
    • Software / Simulation
  • Development
    • Artificial Intelligence
    • Human Robot Interaction / Haptics
    • Mobility / Navigation
    • Research
  • Robots
    • AGVs
    • AMRs
    • Consumer
    • Collaborative Robots
    • Drones
    • Exoskeletons
    • Industrial
    • Self-Driving Vehicles
    • Unmanned Maritime Systems
  • Markets
    • Agriculture
    • Healthcare
    • Logistics
    • Manufacturing
    • Mining
    • Security
  • Financial
    • Investments
    • Mergers & Acquisitions
    • Earnings
  • Resources
    • Careers
    • COVID-19
    • Digital Issues
    • Publications
      • Collaborative Robotics Trends
      • Robotics Business Review
    • RBR50 Winners 2022
    • Search Robotics Database
    • Videos
    • Webinars / Digital Events
  • Events
    • RoboBusiness
    • Robotics Summit & Expo
    • Healthcare Robotics Engineering Forum
    • DeviceTalks
    • R&D 100
    • Robotics Weeks
  • Podcast
    • Episodes
    • Leave a voicemail