X, formerly known as Google X and now part of Alphabet Inc., yesterday announced its Everyday Robot Project, which is working on robots that are as easy for the average person to use. Advances in sensors, machine learning, and low-cost hardware could enable robots to evolve from specialist machines to general-purpose assistance, said X’s leader.

“For the last few years, my team, The Everyday Robot Project, has been working to see if it’s possible to create robots that can do a range of useful tasks in the messy, unstructured spaces of our everyday lives,” wrote Hans Peter Bronmo, project lead and “chief robot whisperer” at X, in a blog post. “Our moonshot is to see if we can make robots as helpful to people in the physical world as computers now are in the virtual world.”

X’s team was created in 2015 and includes roboticists, computer scientists, researchers, makers, and builders. Their goal is to develop affordable robots that don’t require laborious hand coding for tasks in offices or homes.

Current robots are too rigid in the tasks they can do to be useful in people’s daily lives, said X on the Everyday Robot Project webpage. It has built test robots that combine sensors, a manipulator arm, and a mobile base that can safely navigate around people. As Alphabet‘s “skunkworks,” X is also working on self-driving cars, smart glasses, delivery drones, and energy kites.

Developing robot cognition

A major challenge to general-purpose robots is the need for them to fully comprehend dynamic environments as part of perceiving, deciding, and acting autonomously and without painstaking prior programming for every situation. Machine learning and simulation in the cloud are key to improving this capability, said X.

“Where humans naturally combine seeing, understanding, navigating, and acting to move around and achieve their goals, robots typically need careful instruction and coding to do each of these things,” Bronmo said. “This is why it quickly gets very complicated for robots to perform tasks we find easy in highly changeable environments.”

“In the early days of our project, we did this work in the lab, collaborating with teams at Google AI,” he said. “Our tests showed that by giving robots simple tasks and then having them practice, it is indeed possible to teach them to develop new and better capabilities.”

“We investigated how robots can learn from human demonstration, from shared experience, and how we can accelerate learning by simulating robots in the cloud,” said Bronmo. “Once we saw what was possible, we began plotting our path out of the lab and into the real world to test their skills on useful tasks and to see if they could do them reliably and repeatedly.”

Everyday Robot Project refines recycling robots

Bronmo said that the Everyday Robot Project has used different machine learning techniques including simulation, reinforcement learning, and collaborative learning to teach robots to sort recyclables in X’s offices over the past several months. One advantage of simulation is that lessons don’t have to be pre-programmed, and they can be shared among multiple physical robots, wrote Bronmo.

“Each night, tens of thousands of virtual robots practice sorting the waste in a virtual office in our cloud simulator; we then move the training to real robots to refine their sorting ability,” he said. “This real-world training is then integrated back into the simulated training data and shared back with the rest of the robots so that the experience and learning of each robot is shared with them all.”

While building an interesting robot is not the point of the Everyday Robot Project — making service robots more useful is, Bronmo said — the sorting robots reduced the amount of mixed or contaminated waste from 20% to 5%. This addressed a real-world challenge that other recycling robots such as those from AMP Robotics Corp. are solving.

Generalizing lessons

For generalization, a robot must be able to apply what it has learned to another unrelated task, said Bronmo. He acknowledged that getting robots to do this will be difficult.

“Our next challenge is to see if we can take what the robot learned in this task and apply that learning to another task without rebuilding the robot or writing a ton of code from scratch,” he said. “This could prove to be impossible, but we’ll give it a shot.”

Other companies are working on helping robots to learn using the cloud, machine learning and AI, simulation, and 5G, including CloudMinds, Diligent Robotics, Robust AI, and Toyota Research Institute.

“We are years away from a future where robots are helping people every day with everyday tasks,” Bronmo said. “However, results from our recent experiments suggest that we might just be on track.”