The novel coronavirus pandemic has renewed interest in telepresence, which has been a niche application for service robots for several years. The $10 million ANA Avatar XPRIZE, which is intended to help advance telepresence technology, was launched in March 2018, but it has acquired extra relevance now.

For instance, the Japanese government recently announced its Moonshot Research & Development Program, which aims to create remote-controlled avatars to multiply the productivity of the country’s workforce. Japan has been a leader in robotics, in part because of a robot-friendly popular culture and the need to serve and augment an aging population, so avatars are a natural extension of technology.

The ANA Avatar XPRIZE “aims to create an avatar system that can transport human presence to a remote location in real time,” according to the competition’s Web site. In March 2020, there was a summit for the 77 qualified teams, and the organization announced the formation of the XPRIZE Pandemic Alliance in response to the COVID-19 crisis.

Avatar XPRIZE competition continues

“The teams have been progressing through the qualifying submission deadline last October, and there are a few opportunities for teams to join late if they can show that they can make the jump to the semifinals,” said Jacquelyn Morie, co-founder of the University of Southern California’s Institute for Creative Technologies and technical adviser at XPRIZE. “The team summit in March was going to be in Los Angeles, and it went virtual. Teams will submit videos to show where they are as part of testing.”

“We’ve been working with nine expert judges to determine tasks and criteria, and we will expand the pool for semifinals and finals testing,” she told The Robot Report. “Version 1 of the rules is out, and we’re working on Version 2, which will have more details on the scenarios the avatar robots will have to go through on location in semifinals testing in June 2021.”

“Most robotics competitions are about what they can accomplish in autonomous mode,” Morie said. “This one is not about such capabilities but about how that robot functions as a ‘teleportation’ device for providing a sense of presence in a remote location.”

“We’re not trying to test whether a robot can walk up stairs or open a door, but how much operators feel like they’re interacting with other people and the environment,” she added. “This requires a physical presence — a purely digital avatar can never hug your grandmother or pick up debris in a disaster-relief operation.”

AR/VR and haptics for control

XPRIZE typically does not specify how teams can meet challenge goals, and the ANA Avatar XPRIZE is open to technologies such as augmented and virtual reality (AR/VR).

“We don’t prescribe the form factor, but the most logical way to control a robot is through immersive technologies,” said Morie. “If the user can put on a haptic suit and move his or her hands in space, this would move the robot’s hands. This makes sense intuitively, because you’re not making users learn how to do things; they just adjust their motions as a human being would.”

“For seeing what a robot avatar sees, a head-mounted or AR display makes sense for a direct line of sight,” she added. “That’s the best technology we have now.”

“Haptics is one of the key technologies that we’re hoping to push, and almost every team had some kind of haptics,” Morie said. “Getting valid sensations to come back to the operator leads to really interesting applications. If I was homebound but could go to the Grand Canyon and smell the flowers and feel the wind on my face, that would be amazing.”

Exploring human-machine interaction

“Both ends of human-machine interaction will need consideration for future applications,’ Morie acknowledged. “Trust is going to be critical to the adoption of avatar-type robots. We need two-way recognition and a feedback loop for human-to-human connection via avatars.”

“We’ve evolved to recognize intent in certain things, such as where eyes are looking,” she said. “Eventually, some robots will get softer and more human-like to adapt to social uses.”

The ANA Avatar XPRIZE also asked science fiction authors to speculate on what the future might bring for an anthology on telepresence. What about using avatars, not only to give people remote operation, but also to provide superhuman capabilities with robots not constrained by human form or size, such as for construction or handling hazardous materials?

“So far, we’ve looked at a couple of basic categories of needs — social, business, and travel,” said Morie. “We don’t yet know where it will gel, but there are lots of use cases where advances in robotic technology can be aided by human connections.”

“A general-purpose robot is a big challenge — this is a step in the direction of human-to-human connection,” she said. “We’re at the beginning of the exploration. The purpose of the challenge is to push the needle on the technology. The research may inform a new generation of robots.”

Ride-sharing avatar robots

“We can envision a time when we’d have banks of robots like you see banks of scooters today,” Morie saiid. “People will be able to rent the appropriate type for a given function, such as construction or caring for a sick relative.”

5G communications will be helpful for reducing latency over long distances, but the ANA Avatar XPRIZE will use the same predetermined network for evaluating every team, Morie said.

Progress and partners

“The teams are hunkering down, deciding on wheels versus legs, visual systems, and haptics,” explained Morie. “I am impressed with how much people are integrating different technologies. More than half of the qualifying teams are focusing on the sense of smell, which I didn’t expect.”

“SLAM [simultaneous localization and mapping] and lidar are important not just for safe navigation, but also for immersing avatar users in a digital environment,” she said. “Like good sound design, gear is needed for scanning and conveying a particular environment. That technology is currently limited in availability and is cumbersome and expensive. With AR and VR hot for tourism and reconstructing historical sites, we’ll see more and more of that.”

“We encourage teams to find sponsors who’ll help them get further along — we know it’s expensive to develop new robots,” Morie said. “Investors should also be looking at these teams, as robotics is a growing industry. Someday, we’ll look back and see that avatars started with the XPRIZE.”