SecondHands consortium announces progress in proactive humanoid robot

In most cases, “collaborative robots” refers to cobot arms or autonomous mobile robots that are designed to operate safely around humans. A number of projects have been developing other robots that are designed to work closely with people, including the European Union’s SecondHands humanoid service robot project.

This month, the EU Horizon2020 SecondHands consortium announced the “completion” of its ARMAR-6 humanoid platform after five years of development. The robot is intended to help workers in spaces designed for humans in factories and warehouses. By providing a second pair of hands, ARMAR-6 can lift heavy objects, pass tools, or hold a ladder, enabling employees to focus on higher-value tasks, according to the organization.

The Karlsruhe Institute of Technology (KIT) led the design and development, with participation from Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland; Sapienza, University of Rome; and University College London (UCL). The consortium also includes Ocado Technology, which is developing artificial intelligence, robotics, Internet of Things, simulation software, and other technologies for U.K. grocer Ocado Group PLC.

“Robots with sophisticated manipulation, interaction, and learning abilities, such as ARMAR-6, will provide a second pair of hands to people in need of help at home and at work,” stated Tamiim Asfour, a professor at KIT. “The achievements in the project are important steps towards building humanoid robots with embodied intelligence.”

ARMAR-6 includes new sensing, grasping techniques

Many humanoid robots have been relegated to social use cases, but the latest bipedal and two-armed designs have benefited from advances in sensing, AI, and manipulation.

ARMAR-6’s head includes five cameras, and the SecondHands consortium claimed it has made breakthroughs in integrating sensorimotor skills, learning, and reasoning abilities.

“Scientists in different areas of AI often work in isolation, so designing vision algorithms for ARMAR 6 has given us the chance for close collaborations with top roboticists and has pushed us out of our comfort zone towards increasing the robustness and performance of our algorithms,” said Lourdes Agapito, a professor at UCL.

For advanced manipulation, the robot has two arms with eight degrees of freedom (DoF) each, under-actuated hands for small objects, and a combination of visual and haptic sensing.

Machine learning advances in ARMAR-6

ARMAR-6 also uses both data-driven and model-based machine learning approaches, said the SecondHands consortium. It uses new algorithms for gathering knowledge, including activities and actions sequence recognition, tools segmentation, and context classification.

The robot has new algorithms to build geometric and semantic 3D maps of an environment that accommodates moving objects. It also uses “supervised and unsupervised approaches for 3D reconstruction of human poses and semantic scene understanding that can learn from just 2D labels or even no labels at all,” said the consortium. This helps the robot determine human direction and intent without extensive data annotation.

SecondHands focuses on human-machine interaction

One of the biggest challenges in autonomous systems is the handoff to human colleagues or supervisors. The SecondHands consortium said that ARMAR-6 can learn from exploration on addition to teaching or coaching by humans. The android is able to “infer when a human needs help and proactively offer appropriate assistance,” said the consortium.

With neural networks for speech recognition, dialog modeling, and speech synthesis, the SecondHands project is working toward more natural-language interactions between humans and robots. This will help both usability and adoption, it said.

“The breakthroughs made in areas like natural language interfaces and task understanding will lead to better acceptance of cobots by humans, and allow them to be used in a more easy, natural way,” said Dr. Sebastian Stüker of  KIT. “This is an important building block that will contribute to the emergence of true collaboration.”

The robot also has a state-dependent dynamic system and reactive motion planning for safe operations around people. The consortium and other researchers have studied how the robot can grasp and manipulate large objects by following a human’s lead.

“The breakthroughs demonstrated in handover scenarios, guard removal and insertion, guard co-manipulation, and obstacle avoidance will play a crucial role in having reliable robots that can be used in everyday scenarios,” asserted Aude Billard, a professor at EPFL.

SecondHands tests humanoid robot at Ocado

Ocado tested the ARMAR-6 at one of its facilities. The company, which also builds and uses automated storage and retrieval systems (AR/RS) and pick-and-place robotics, raised $673.3 million last December after a fire destroyed one of its warehouses. Ocado is also building highly automated customer fulfillment centers (CFC) with The Kroger Co. in Georgia and Florida.

“It was really valuable to be able to test in the Ocado warehouse as a real-world environment,” said Agapito. “The dynamic, constantly changing conditions tested our vision, robotics, and language-processing algorithms to go beyond the current state of the art.”

“The challenging real-world environment of the Ocado CFC allowed us to develop a reasoning system and perception algorithms to provide support in industrial maintenance tasks,” said Fiora Pirri, a professor at University La Sapienza.

Although Ocado acknowledged that it has no plans to deploy the robot in its warehouses, ARMAR-6’s humanoid design and machine learning capabilities could make it useful for other applications.

“Humanoid robots are key for improving flexibility and safety in industrial contexts in a way that is genuinely useful,” said Dr. Graham Deacon, a robotics research fellow at Ocado Technology. “The same technologies which enable the ARMAR-6 to communicate and interact with humans, like natural language comprehension, soft manipulation, and 3D spatial awareness, also mean the robot could be developed further to help in other situations, like in helping to reduce contamination or in assisted living.”