Application of Physiological Data in Human-Robot Interaction with Anthropomorphic Service Robots: Affect-HRI Dataset

Background and Significance

In interactions between humans and humans, as well as humans and robots, the interacting entity can influence human emotional states. Unlike humans, robots inherently cannot exhibit empathy and thus cannot mitigate adverse emotional reactions. To create a responsible and empathetic human-robot interaction system, especially one involving anthropomorphic service robots, it is crucial to understand how robot behavior affects human emotions. To this end, researchers have introduced a new comprehensive dataset called Affect-HRI, which for the first time includes physiological data labeled with human emotions (i.e., mood and feelings), collected during an ongoing human-robot interaction study.

Study Source

The study was authored by Judith S. Heinisch, Jérôme Kirchhoff, Philip Busch, Janine Wendt, Oskar von Stryk, and Klaus David from the University of Kassel and the Technical University of Darmstadt. The paper is published in the journal “Scientific Data,” Volume 11, Number 333, in 2024.

Study Procedure

Research Design and Methods

The study employed a mixed-methods approach and a between-subjects design to investigate the impact of responsible interaction with anthropomorphic service robots on human emotions. Two anthropomorphic service robots, Tiago++ and Elenoide, were selected for the study. The robots simulated five different behavioral conditions: neutral, transparent, responsible, ethical, and unethical.

Experimental Scenario Setup

The experiment was conducted in a simulated retail environment featuring three scenarios: product consultation, customer account creation, and mold remover handover. Participants were required to interact with the robots in these scenarios, with robots responding and operating based on predefined scripts and conditions.

Data Collection

During the experiment, participants wore Empatica E4 wristbands to collect physiological data, including Galvanic Skin Response (GSR), heart rate (HR), and skin temperature (ST). Additionally, participants completed questionnaires before and after the experiment to record their emotional states and evaluations of the robots.

Data Analysis

The dataset includes physiological sensor data, questionnaire data, and robot behavior data (speech and gestures). Through analysis of this data, researchers were able to assess the impact of robot behavior on participants’ emotional states under different conditions.

Study Results

Key Findings

1. Emotional Changes: Under the transparency condition, participants’ negative emotional reactions to data privacy issues were alleviated, leading to a more relaxed and positive overall emotional state. Conversely, in the responsibility condition, failed handover tasks significantly decreased participants’ pleasure and increased tension.
2. Impact of Ethical and Unethical Behavior: Compared to the neutral condition, ethical behavior by the robots lowered participants’ tension, making their emotional states more relaxed. Unethical behavior led to increased tension and unpleasant emotions.

Conclusion

This study provides the first physiological dataset labeled with human emotions for use in affective computing in human-robot interaction. The dataset can be used not only to validate existing emotion recognition methods but also to develop new emotion recognition technologies. The findings show that the transparency and ethicality of robot behavior significantly affect human emotional states, offering important guidance for designing more empathetic and responsible service robots in the future.

Study Highlights

1. Comprehensive Dataset: The Affect-HRI dataset is the first publicly available human-robot interaction dataset containing physiological data labeled with emotions.
2. Multi-Method Research Design: The study combines expertise from psychology, computer science, and law, providing a holistic view of human-robot interaction.
3. Realistic Experimental Scenarios: The study was conducted in a simulated retail environment, making the scenario settings close to the real world, enhancing the applicability of the findings.

Scientific and Application Value

1. Scientific Value: The study provides valuable data resources and new perspectives for affective computing and human-robot interaction research, helping to advance and refine related technologies.
2. Application Value: The findings can be applied to design more human-centered and responsible service robots, improving user experience and acceptance. Additionally, the dataset can be used in cross-disciplinary research, such as law and ethics, to explore issues of responsibility and transparency in human-robot interaction.

Additional Information

Participants and Recruitment

The experiment recruited 175 participants, of which 29 were excluded due to technical issues or failure to complete tasks as required. The final valid sample consisted of 146 participants (85 females, 60 males, 1 gender unknown) aged from 18 to 66 years. The participants were mainly students and staff from the Technical University of Darmstadt.

Data Anonymization

To protect participant privacy, all data were anonymized. The experimental data underwent multi-layer anonymization, including time anonymization and participant ID anonymization, ensuring its secure and compliant use in the study.

Dataset Validation

The dataset underwent rigorous technical validation and quality control to ensure data accuracy and reliability. Researchers used various analytical tools and methods for comprehensive validation and assessment.

Usage Suggestions

The dataset can be used for researching and improving emotion recognition methods, evaluating the impact of robot behavior on user experience, and exploring issues of responsibility and transparency in human-robot interaction. Researchers recommend using programming languages like Python for data processing and analysis and combining it with other open-source datasets for more in-depth research.

Conclusion

The Affect-HRI dataset provides a valuable resource for the fields of affective computing and human-robot interaction. The study’s findings reveal significant effects of robot behavior on human emotional states, offering important references for designing more empathetic and responsible service robots. Through interdisciplinary research, the dataset can also be used to explore ethical and legal issues in human-robot interaction, promoting advances in related fields.