# How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Survey

arXiv: https://arxiv.org/abs/2404.00938 · HTML: https://mlsanity.com/paper/2404.00938/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 35 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Understanding Social Reasoning in Language Models with Language Models](https://mlsanity.com/paper/2306.15448/) (2023)
- [MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria](https://mlsanity.com/paper/2311.13951/) (2023)
- [VITA: A Multi-modal LLM-based System for Longitudinal, Autonomous, and Adaptive Robotic Mental Well-being Coaching](https://mlsanity.com/paper/2312.09740/) (2023)
- [Knowledge-Grounded Dialogue Flow Management for Social Robots and Conversational Agents](https://mlsanity.com/paper/2108.02174/) (2021)
- [Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism](https://mlsanity.com/paper/1311.0352/) (2013)
- [A Sign Language Recognition System with Pepper, Lightweight-Transformer, and LLM](https://mlsanity.com/paper/2309.16898/) (2023)
- [GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System](https://mlsanity.com/paper/2306.01741/) (2023)
- [A Systematic Literature Review of Experiments in Socially Assistive Robotics using Humanoid Robots](https://mlsanity.com/paper/1711.05379/) (2017)

## Built on by (most cited first, 4 of 4)

- [Understanding Persuasive Interactions between Generative Social Agents and Humans: The Knowledge-based Persuasion Model (KPM)](https://mlsanity.com/paper/2602.11483/) (2026)
- [LLM Granularity for On-the-Fly Robot Control](https://mlsanity.com/paper/2406.14653/) (2024)
- [An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Robotics](https://mlsanity.com/paper/2406.06400/) (2024)
- [The Supportiveness-Safety Tradeoff in LLM Well-Being Agents](https://mlsanity.com/paper/2602.04487/) (2026)

## Abstract (arXiv)

Socially assistive robots (SARs) have shown great success in providing personalized cognitive-affective support for user populations with special needs such as older adults, children with autism spectrum disorder (ASD), and individuals with mental health challenges. The large body of work on SAR demonstrates its potential to provide at-home support that complements clinic-based interventions delivered by mental health professionals, making these interventions more effective and accessible. However, there are still several major technical challenges that hinder SAR-mediated interactions and interventions from reaching human-level social intelligence and efficacy. With the recent advances in large language models (LLMs), there is an increased potential for novel applications within the field of SAR that can significantly expand the current capabilities of SARs. However, incorporating LLMs introduces new risks and ethical concerns that have not yet been encountered, and must be carefully be addressed to safely deploy these more advanced systems. In this work, we aim to conduct a brief survey on the use of LLMs in SAR technologies, and discuss the potentials and risks of applying LLMs to the following three major technical challenges of SAR: 1) natural language dialog; 2) multimodal understanding; 3) LLMs as robot policies.