# Robot guide with multi-agent control and automatic scenario generation with LLM

arXiv: https://arxiv.org/abs/2509.10317 · HTML: https://mlsanity.com/paper/2509.10317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 12 Sep 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity](https://mlsanity.com/paper/2009.02119/) (2020)
- [Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots](https://mlsanity.com/paper/1810.12541/) (2018)
- [A Survey of Behavior Trees in Robotics and AI](https://mlsanity.com/paper/2005.05842/) (2020)
- [LLM-BRAIn: AI-driven Fast Generation of Robot Behaviour Tree based on Large Language Model](https://mlsanity.com/paper/2305.19352/) (2023)
- [BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs](https://mlsanity.com/paper/2403.12761/) (2024)
- [Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review](https://mlsanity.com/paper/2502.11518/) (2025)
- [Developing Social Robots with Empathetic Non-Verbal Cues Using Large Language Models](https://mlsanity.com/paper/2308.16529/) (2023)
- [Artificial Emotional Intelligence in Socially Assistive Robots for Older Adults: A Pilot Study](https://mlsanity.com/paper/2201.11167/) (2022)
- [Decentralized Task and Path Planning for Multi-Robot Systems](https://mlsanity.com/paper/2011.10034/) (2020)
- [Deep Gesture Generation for Social Robots Using Type-Specific Libraries](https://mlsanity.com/paper/2210.06790/) (2022)
- [Ain't Misbehavin' -- Using LLMs to Generate Expressive Robot Behavior in Conversations with the Tabletop Robot Haru](https://mlsanity.com/paper/2402.11571/) (2024)
- [iCub Detecting Gazed Objects: A Pipeline Estimating Human Attention](https://mlsanity.com/paper/2308.13318/) (2023)
- [Upgrading Pepper Robot s Social Interaction with Advanced Hardware and Perception Enhancements](https://mlsanity.com/paper/2409.01036/) (2024)
- [The Effect of Robot Posture and Idle Motion on Spontaneous Emotional Contagion during Robot-Human Interactions](https://mlsanity.com/paper/2209.00983/) (2022)

## Abstract (arXiv)

The article describes the development of a hybrid social robot control architecture to overcome the limitations of traditional approaches, where behavior scripts manually synchronize the robot's actions and text, and existing methods focus primarily on short dialogue responses. The architecture of the proposed system combines a multi-agent resource management system with automatic generation of behavior scenarios based on large language models. This system automates the preparation of text and commands for the robot's non-verbal behavior for extended narratives and resolves resource conflicts between multiple execution mechanisms. The system was tested on the MENTOR-1 tour guide robot, for which it successfully generated scenarios automatically and demonstrated more natural and rich behavior compared to existing approaches. The proposed approach provides full automation of both scenario preparation and execution through efficient resource management, enhancing the quality of social robot interaction in long-term storytelling tasks.