When the Cook Makes a Mistake: Analyzing Communication Errors and (In)Formal Repair Strategies in Human-Robot Dialogues.

Given the increased capabilities of large language models, researchers in the field of human robot interaction (HRI) are investigating methods to implement such models into social robots. Despite this rapid evolution, social robots still struggle to pick up basic social cues and detect when they make communication mistakes. Although this crucial skill is trivial for humans, a lack of it causes a degradation of trust between human and robots, which is reflected in shortened or terminated interactions. Moreover, even if such mistakes are recognized, it is unclear which strategies should best be applied to repair trust in human-robot interactions.

To address these issues, we conducted a Wizard of Oz experiment with 40 participants. In a Wizard of Oz experiment, computer systems are either partially or fully controlled by a human operator (i.e., “wizard”). We used the social robot Furhat and fully scripted six dialogues that were each linked to a culinary recipe. In each dialogue, we inserted four different communication mistakes: interruption, oversharing information, undersharing information, and irrelevant comments (Chakraborti et al., 2017; Serholt et al., 2020). While we knew the exact positions of the mistakes, we wanted to check whether participants also perceived them as ‘erroneous’. To this end, we asked the participants for binary feedback after each reply: good/bad robot turn. The feedback was asked through human intervention: the wizard sent a rumble (i.e., vibration) to a PlayStation controller; once the participant’s feedback was received, the operator had to manually start the next robot turn. We asked participants to explain why they found a turn erroneous after each dialogue.

Besides our focus on communication mistakes, we also wanted to investigate how such mistakes can best be addressed. We therefore looked at three repair strategies: apology, promise, explanation (Esterwood & Robert, 2023). We included formality as an additional binary variable, since we noticed that humans who design robot conversational flows often come up with quite formal and lengthy replies, while colloquial answers are another option that intuitively makes sense too (Torrey et al., 2013). Each dialogue/recipe was thus linked to a repair strategy and a degree of formality: a formal brownie-bot that makes promises vs. an informal soup-bot explaining its mistakes. After each dialogue, participants were given a questionnaire in which they rated the robot. Moreover, at the end of the experiment an additional questionnaire was provided to evaluate their overall experience and compare the different systems.

We are currently conducting an in-depth data analysis along the following research questions: (1) Which communication mistakes are most/least disturbing? (2) Which repair strategies work best? (3) What is the effect of formality in human-robot interactions? For the first question, participants indicated oversharing of information as the least annoying communication mistake. For the second question, we found that an informal explanation was one of the most popular repair strategies, while participants generally disliked both formal and informal promises. Finally, participants said that they preferred the informal system over the formal system.