Beyond Static Instruction: A Multi-agent AI Framework for Adaptive Augmented Reality Robot Training
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Augmented Reality (AR) offers powerful visualization capabilities for industrial robot training, yet current interfaces remain predominantly static, failing to account for learners' diverse cognitive profiles. In this paper, we present an AR application for robot training and propose a multi-agent AI framework for future integration that bridges the gap between static visualization and pedagogical intelligence. We report on the evaluation of the baseline AR interface with 36 participants performing a robotic pick-and-place task. While overall usability was high, notable disparities in task duration and learner characteristics highlighted the necessity for dynamic adaptation. To address this, we propose a multi-agent framework that orchestrates multiple components to perform complex preprocessing of multimodal inputs (e.g., voice, physiology, robot data) and adapt the AR application to the learner's needs. By utilizing autonomous Large Language Model (LLM) agents, the proposed system would dynamically adapt the learning environment based on advanced LLM reasoning in real-time.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 1 of its 20 references that are in the map. First posted on arXiv 31 Jan 2026.
Builds on
1 of 20 references.
References · 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistance
- 2023Augmenting Learning with Augmented Reality: Exploring the Affordances of AR in Supporting Mastery of Complex Psychomotor Tasks
- 2024An Interactive Agent Foundation Model
- 2026Explainable OOHRI: Communicating Robot Capabilities and Limitations as Augmented Reality Affordances ↗
- 2025MaP-AVR: A Meta-Action Planner for Agents Leveraging Vision Language Models and Retrieval-Augmented Generation
- 2025An Embodied AR Navigation Agent: Integrating BIM with Retrieval-Augmented Generation for Language Guidance
- 2025Visual Agentic Reinforcement Fine-Tuning
- 2025PFEA: An LLM-based High-Level Natural Language Planning and Feedback Embodied Agent for Human-Centered AI