# Beyond Static Instruction: A Multi-agent AI Framework for Adaptive Augmented Reality Robot Training

arXiv: https://arxiv.org/abs/2603.00016 · HTML: https://mlsanity.com/paper/2603.00016/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 1 of its 20 references that are in the map.
- First posted on arXiv 31 Jan 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 1 of 1)

- [Mixed Reality Outperforms Virtual Reality for Remote Error Resolution in Pick-and-Place Tasks](https://mlsanity.com/paper/2502.06141/) (2025)

## Abstract (arXiv)

Augmented Reality (AR) offers powerful visualization capabilities for industrial robot training, yet current interfaces remain predominantly static, failing to account for learners' diverse cognitive profiles. In this paper, we present an AR application for robot training and propose a multi-agent AI framework for future integration that bridges the gap between static visualization and pedagogical intelligence. We report on the evaluation of the baseline AR interface with 36 participants performing a robotic pick-and-place task. While overall usability was high, notable disparities in task duration and learner characteristics highlighted the necessity for dynamic adaptation. To address this, we propose a multi-agent framework that orchestrates multiple components to perform complex preprocessing of multimodal inputs (e.g., voice, physiology, robot data) and adapt the AR application to the learner's needs. By utilizing autonomous Large Language Model (LLM) agents, the proposed system would dynamically adapt the learning environment based on advanced LLM reasoning in real-time.