# ARROCH: Augmented Reality for Robots Collaborating with a Human

arXiv: https://arxiv.org/abs/2109.10400 · HTML: https://mlsanity.com/paper/2109.10400/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2021.

## Built on by (most cited first, 5 of 5)

- [Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together](https://mlsanity.com/paper/2202.01493/) (2022)
- [Real-Time Operator Takeover for Visuomotor Diffusion Policy Training](https://mlsanity.com/paper/2502.02308/) (2025)
- [BeSimulator: A Large Language Model Powered Text-based Behavior Simulator](https://mlsanity.com/paper/2409.15865/) (2024)
- [Learning Visualization Policies of Augmented Reality for Human-Robot Collaboration](https://mlsanity.com/paper/2211.07028/) (2022)
- [fARfetch: Enabling Collocated AR-HRC in Large Visually Diverse Environments with VLM-Driven AR Content Adaptation](https://mlsanity.com/paper/2606.25162/) (2026)

## Abstract (arXiv)

Human-robot collaboration frequently requires extensive communication, e.g., using natural language and gestures. Augmented reality (AR) has provided an alternative way of bridging the communication gap between robots and people. However, most current AR-based human-robot communication methods are unidirectional, focusing on how the human adapts to robot behaviors, and are limited to single-robot domains. In this paper, we develop AR for Robots Collaborating with a Human (ARROCH), a novel algorithm and system that supports bidirectional, multi-turn, human-multi-robot communication in indoor multi-room environments. The human can see through obstacles to observe the robots' current states and intentions, and provide feedback, while the robots' behaviors are then adjusted toward human-multi-robot teamwork. Experiments have been conducted with real robots and human participants using collaborative delivery tasks. Results show that ARROCH outperformed a standard non-AR approach in both user experience and teamwork efficiency. In addition, we have developed a novel simulation environment using Unity (for AR and human simulation) and Gazebo (for robot simulation). Results in simulation demonstrate ARROCH's superiority over AR-based baselines in human-robot collaboration.