# To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions

arXiv: https://arxiv.org/abs/2403.12533 · HTML: https://mlsanity.com/paper/2403.12533/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Builds on (most cited first, 23 of 23)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Tool Learning with Foundation Models](https://mlsanity.com/paper/2304.08354/) (2023)
- [ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph](https://mlsanity.com/paper/2403.00839/) (2024)
- [Understanding Large-Language Model (LLM)-powered Human-Robot Interaction](https://mlsanity.com/paper/2401.03217/) (2024)
- [CoPAL: Corrective Planning of Robot Actions with Large Language Models](https://mlsanity.com/paper/2310.07263/) (2023)
- [Large Language Models as Zero-Shot Human Models for Human-Robot Interaction](https://mlsanity.com/paper/2303.03548/) (2023)
- [NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants](https://mlsanity.com/paper/2301.05223/) (2023)
- [Generative Expressive Robot Behaviors using Large Language Models](https://mlsanity.com/paper/2401.14673/) (2024)
- [Collaborative Storytelling with Large-scale Neural Language Models](https://mlsanity.com/paper/2011.10208/) (2020)
- [LaMI: Large Language Models for Multi-Modal Human-Robot Interaction](https://mlsanity.com/paper/2401.15174/) (2024)
- [Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models](https://mlsanity.com/paper/2309.04316/) (2023)
- [Proactive Robot Assistance via Spatio-Temporal Object Modeling](https://mlsanity.com/paper/2211.15501/) (2022)
- [Improved Trust in Human-Robot Collaboration with ChatGPT](https://mlsanity.com/paper/2304.12529/) (2023)
- [Intention estimation from gaze and motion features for human-robot shared-control object manipulation](https://mlsanity.com/paper/2208.08688/) (2022)
- [A Survey of Multi-Agent Human-Robot Interaction Systems](https://mlsanity.com/paper/2212.05286/) (2022)
- [Two ways to make your robot proactive: reasoning about human intentions, or reasoning about possible futures](https://mlsanity.com/paper/2205.05492/) (2022)
- [Group Dynamics: Survey of Existing Multimodal Models and Considerations for Social Mediation](https://mlsanity.com/paper/2306.17374/) (2023)
- [What Could a Social Mediator Robot Do? Lessons from Real-World Mediation Scenarios](https://mlsanity.com/paper/2306.17379/) (2023)
- [A Study on Social Robot Behavior in Group Conversation](https://mlsanity.com/paper/2312.12473/) (2023)

## Built on by (most cited first, 10 of 10)

- [Affordance-based Robot Manipulation with Flow Matching](https://mlsanity.com/paper/2409.01083/) (2024)
- [LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey](https://mlsanity.com/paper/2505.00753/) (2025)
- [A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy](https://mlsanity.com/paper/2506.09420/) (2025)
- [Tulip Agent -- Enabling LLM-Based Agents to Solve Tasks Using Large Tool Libraries](https://mlsanity.com/paper/2407.21778/) (2024)
- [Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory](https://mlsanity.com/paper/2405.20189/) (2024)
- [RHINO: Learning Real-Time Humanoid-Human-Object Interaction from Human Demonstrations](https://mlsanity.com/paper/2502.13134/) (2025)
- [Graph-Grounded LLMs: Leveraging Graphical Function Calling to Minimize LLM Hallucinations](https://mlsanity.com/paper/2503.10941/) (2025)
- [RAIDER: Tool-Equipped Large Language Model Agent for Robotic Action Issue Detection, Explanation and Recovery](https://mlsanity.com/paper/2503.17703/) (2025)
- [An LLM-powered Natural-to-Robotic Language Translation Framework with Correctness Guarantees](https://mlsanity.com/paper/2508.19074/) (2025)
- [J-ORA: A Framework and Multimodal Dataset for Japanese Object Identification, Reference, Action Prediction in Robot Perception](https://mlsanity.com/paper/2510.21761/) (2025)

## Abstract (arXiv)

How can a robot provide unobtrusive physical support within a group of humans? We present Attentive Support, a novel interaction concept for robots to support a group of humans. It combines scene perception, dialogue acquisition, situation understanding, and behavior generation with the common-sense reasoning capabilities of Large Language Models (LLMs). In addition to following user instructions, Attentive Support is capable of deciding when and how to support the humans, and when to remain silent to not disturb the group. With a diverse set of scenarios, we show and evaluate the robot's attentive behavior, which supports and helps the humans when required, while not disturbing if no help is needed.