# Understanding Large-Language Model (LLM)-powered Human-Robot Interaction

arXiv: https://arxiv.org/abs/2401.03217 · HTML: https://mlsanity.com/paper/2401.03217/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jan 2024.

## Built on by (most cited first, 30 of 43)

- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use](https://mlsanity.com/paper/2510.04550/) (2025)
- [Safety Guardrails for LLM-Enabled Robots](https://mlsanity.com/paper/2503.07885/) (2025)
- [ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning](https://mlsanity.com/paper/2406.19741/) (2024)
- [Bi-VLA: Vision-Language-Action Model-Based System for Bimanual Robotic Dexterous Manipulations](https://mlsanity.com/paper/2405.06039/) (2024)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions](https://mlsanity.com/paper/2403.12533/) (2024)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Multi-Agent Systems for Robotic Autonomy with LLMs](https://mlsanity.com/paper/2505.05762/) (2025)
- [GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation](https://mlsanity.com/paper/2512.17495/) (2025)
- [Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory](https://mlsanity.com/paper/2405.20189/) (2024)
- [EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning](https://mlsanity.com/paper/2410.23234/) (2024)
- [HistNERo: Historical Named Entity Recognition for the Romanian Language](https://mlsanity.com/paper/2405.00155/) (2024)
- [Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineering](https://mlsanity.com/paper/2506.06384/) (2025)
- [Misalignment or misuse? The AGI alignment tradeoff](https://mlsanity.com/paper/2506.03755/) (2025)
- [Prediction-Powered Communication with Distortion Guarantees](https://mlsanity.com/paper/2509.24373/) (2025)
- [Understanding Persuasive Interactions between Generative Social Agents and Humans: The Knowledge-based Persuasion Model (KPM)](https://mlsanity.com/paper/2602.11483/) (2026)
- [A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference](https://mlsanity.com/paper/2606.15453/) (2026)
- [The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare](https://mlsanity.com/paper/2411.03287/) (2024)
- [Encoding Inequity: Examining Demographic Bias in LLM-Driven Robot Caregiving](https://mlsanity.com/paper/2503.05765/) (2025)
- [Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models](https://mlsanity.com/paper/2503.15491/) (2025)
- [Building Knowledge from Interactions: An LLM-Based Architecture for Adaptive Tutoring and Social Reasoning](https://mlsanity.com/paper/2504.01588/) (2025)
- [An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Robotics](https://mlsanity.com/paper/2406.06400/) (2024)
- [GenComUI: Exploring Generative Visual Aids as Medium to Support Task-Oriented Human-Robot Communication](https://mlsanity.com/paper/2502.10678/) (2025)
- [When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemma](https://mlsanity.com/paper/2503.07320/) (2025)
- [Plan-and-Act using Large Language Models for Interactive Agreement](https://mlsanity.com/paper/2504.01252/) (2025)
- [Comparing Self-Disclosure Themes and Semantics to a Human, a Robot, and a Disembodied Agent](https://mlsanity.com/paper/2504.06374/) (2025)
- [SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs](https://mlsanity.com/paper/2512.04868/) (2025)
- [Knowledge-Based Design Requirements for Generative Social Robots in Higher Education](https://mlsanity.com/paper/2602.12873/) (2026)
- [A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems](https://mlsanity.com/paper/2604.01179/) (2026)

## Abstract (arXiv)

Large-language models (LLMs) hold significant promise in improving human-robot interaction, offering advanced conversational skills and versatility in managing diverse, open-ended user requests in various tasks and domains. Despite the potential to transform human-robot interaction, very little is known about the distinctive design requirements for utilizing LLMs in robots, which may differ from text and voice interaction and vary by task and context. To better understand these requirements, we conducted a user study (n = 32) comparing an LLM-powered social robot against text- and voice-based agents, analyzing task-based requirements in conversational tasks, including choose, generate, execute, and negotiate. Our findings show that LLM-powered robots elevate expectations for sophisticated non-verbal cues and excel in connection-building and deliberation, but fall short in logical communication and may induce anxiety. We provide design implications both for robots integrating LLMs and for fine-tuning LLMs for use with robots.