Use of Winsome Robots for Understanding Human Feedback (UWU)
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
As social robots become more common, many have adopted cute aesthetics aiming to enhance user comfort and acceptance. However, the effect of this aesthetic choice on human feedback in reinforcement learning scenarios remains unclear. Previous research has shown that humans tend to give more positive than negative feedback, which can cause failure to reach optimal robot behavior. We hypothesize that this positive bias may be exacerbated by the robot's level of perceived cuteness. To investigate, we conducted a user study where participants critique a robot's trajectories while it performs a task. We then analyzed the impact of the robot's aesthetic cuteness on the type of participant feedback. Our results suggest that there is a shift in the ratio of positive to negative feedback when perceived cuteness changes. In light of this, we experiment with a stochastic version of TAMER which adapts based on the user's level of positive feedback bias to mitigate these effects.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 3 of its 34 references that are in the map. First posted on arXiv 7 Feb 2025.
Builds on
3 of 34 references.
Closest ancestors · 2
Same direction, newest first
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback27 Jul
- 2019Deep Reinforcement Learning from Policy-Dependent Human Feedback12 Feb
Other references · 1
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025CHARM: Considering Human Attributes for Reinforcement Modeling
- 2024Are Large Language Models Aligned with People's Social Intuitions for Human-Robot Interactions?
- 2025HAPI: A Model for Learning Robot Facial Expressions from Human Preferences
- 2024Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
- 2026Rethinking the Implications of Human Feedback for Preference Learning in Human-Robot Collaboration
- 2020The EMPATHIC Framework for Task Learning from Implicit Human Feedback
- 2026AI Should Not Only Be Helpful. It Should Be Contingent. Artificial Intimacy, Sycophancy, and the Future of Social Learning
- 2024Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind