Mehdi Hellou

Personalised conversation agents for long-term human-robot interaction

During my final year at the graduate school of engineering Polytech Sorbonne, I carried out an internship of six months at Softbank Robotics Europe. This international company created the robots Pepper (https://www.softbankrobotics.com/emea/en/pepper) and Nao (https://www.softbankrobotics.com/emea/en/nao).

My project’s topic consisted of improving the interaction between Pepper and human by personalising its behaviours and speech for long-term interaction. This project sought to answer the main issue present in most of the interactions we can have with a humanoid robot: its repetitive behaviours resulting in the decreasing of user engagement.

Hence, a robot that could adapt its behaviours and speech to the user’s personality, preferences, needs and share memories with the users, it would allow us to develop and enhance in a better way the social interactions we could have with robots. In this way, we illustrate the notion of “personalization” for companion robots.

Under the supervision of Bahar Irfan, PhD student at Plymouth University, and Alexandre Mazel, Innovation Director at Softbank Robotics Europe, I worked for six months in this project to personalize the behaviour and speech of the robot Pepper for goal-oriented dialogues. We put the scenario of a barista robot, where Pepper had to take a customer’s order. It also could remember regular customers and to propose them once again their regulars orders. For that purpose, we employed a multi-modal incremental Bayesian Network (*Irfan et al. 2018 “Multi-modal Open-Set Person Identification in HRI.”) to recognize the users’ identity. For the robot’s speech, we used a rules-based dialog model built from scratch.

As part of the internship, a one-week experiment was carried out at Cité Universitaire (17 Boulevard Jourdan, 75014 Paris) to assess the overall project. A Late-Breaking Report about it was published at the HRI 2020 conference at Cambridge University ¹.