Designers of service robots must marry advanced technologies such as natural language processing, facial recognition, and a humanoid form factor with an understanding of psychology for human-machine interaction. SoftBank Robotics Group Corp.’s Pepper, which is being used in retail, educational, and hospitality settings, is an example of the thought around trust that goes into building and using such robots.

“We’re currently working on several projects, with a focus on retail and banking,” said Matt Willis, design and human-robot interaction strategy lead at SoftBank Robotics. “There are plenty of other robots in retail for tasks other than customer service.”

“While they may be working on other tasks, such as scanning inventory or cleaning, any robot in a public setting requires an understanding of human interaction,” he told The Robot Report. “It must convey its purpose and intent for things such as what direction it’s going.”

Building Pepper with purpose

“Since Pepper is primarily customer-facing, its form factor has a lot of affordances for people,” Willis said. “We include speaking, waving, and other nonverbal gestures.”

“When somebody walks into a store and sees the robot, it grabs their attention,” he added. “People are curious and looking to be entertained, and our responsibility is to convey its purpose and build trust.”

SoftBank Robotics, formerly Aldebaran, makes the humanoid Pepper, Nao, and Romeo robots, as well as the Whiz floor-cleaning robot using BrainOS technology for autonomous navigation.

“Our robots offer different things. Our principle for providing value through robotics includes a human-first approach,” said Willis. “What problem are we trying to solve? Nao has been a great platform for educating the next generation of engineers.”

“Pepper is also used in education and research, but it is larger, with a tablet interface,” he explained. “Public schools in Boston and San Francisco are already using Pepper, and we’ll have more educational news and products soon.”

Multimodal interaction

SoftBank’s hardware and software developers continue to learn from human-robot interactions. Pepper has been in HSBC banks in New York, Seattle, and Beverly Hills, Calif.

“As we focus on how to support retail environments, we see what we can do in a conversational setting to support multimodal interactions, like speech, gesture, and on-screen displays,” Willis said. “In cases where things don’t go as planned, we look to improve our technology and services to support customers so that they still have a great experience.”

“Social cues, such as Pepper nodding, can mean an acknowledgement in one culture and in others, ‘Yes,'” he said. “We manage that through testing to find common responses.”

What about emotion recognition? “Pepper can recognize some states, but the better question is, ‘How can we use that to guide interactions?'” responded Willis. “Emotion is one of many tools to understand how someone is behaving and what they want.”

Working to exceed expectations

“We chose not to make Pepper too human-like,” Willis explained. “Pepper is often the face of social robots, and it has an attractive, non-threatening design.”

“People are more willing to accept occasional failures in voice recognition, based on the design,” he added. “Such robot form factors encourage you to trust and help the robot.”

While SoftBank Robotics has not announced any future humanoid designs, it is working on other improvements, which are pushed out to users via a Robotics-as-a-Service (RaaS) model.

“Pepper’s capabilities are always improving, including internal hardware and services,” said Willis. “As we demonstrated at CES, we’re also integrating with other cloud services to deliver more front-end experiences and value to store associates and consumers.”

“Pepper has been strong for customer engagement and giving information,” he noted. “We’re focusing on retail and education, but we’re always interested in new spaces like healthcare.”

Social robot challenges

Social and other consumer robot makers have struggled in the past year, with Kuri, Jibo, Keecker, and Anki among the casualties. Anki’s shutdown was one of the most-read stories on The Robot Report so far this year.

“SoftBank is leading in this space with Pepper, and we face the same challenges as every social robotics company,” Willis acknowledged. “It’s no longer a challenge of navigating from here to there. The question is moving to what it should do.”

“Our approach is to experiment and try new things and to start with the need and trust,” he said. “We need to balance that with current expectations. If you see a humanoid robot in a store and you’ve never seen one before, you might approach it with a different goal in mind than someone who is going to the store and who has a question for the robot to answer.”

While SoftBank has had slow renewals of its contracts with some retailers, its trial with HSBC continues. “We’re still learning from engagements at their branches,” Willis said. “We’re improving the platform and services.”

AI and funding

In 2017, SoftBank Group, the parent of SoftBank Robotics, purchased Boston Dynamics. While Boston Dynamics and SoftBank Robotics said they aren’t working on any robots together, Boston Dynamics is working to commercialize its Handle and Spot Mini robots.

One company funded by SoftBank Group’s Vision Fund is CloudMinds Technology Inc., which recently filed for a $500 million initial public offering. Its cloud-based AI could help Pepper be more intelligent.

Tokyo-based SoftBank last week announced a second Vision Fund worth $108 billion, so it could help keep Pepper and other robots going as the technologies and markets mature.

“There’s still plenty to learn, and I hope we’ll see many more social robotics and AI companies come up,” Willis said.