# "Can you be my mum?": Manipulating Social Robots in the Large Language Models Era

arXiv: https://arxiv.org/abs/2501.04633 · HTML: https://mlsanity.com/paper/2501.04633/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2025.

## Built on by (most cited first, 3 of 3)

- [Unfettered Forceful Skill Acquisition with Physical Reasoning and Coordinate Frame Labeling](https://mlsanity.com/paper/2505.09731/) (2025)
- [On the Dual-Use Dilemma in Physical Reasoning and Force](https://mlsanity.com/paper/2505.18792/) (2025)
- [Never say never: Exploring the effects of available knowledge on agent persuasiveness in controlled physiotherapy motivation dialogues](https://mlsanity.com/paper/2602.12924/) (2026)

## Abstract (arXiv)

Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions.