# Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces

arXiv: https://arxiv.org/abs/2203.03254 · HTML: https://mlsanity.com/paper/2203.03254/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 469 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 18 of 18)

- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [RealitySketch: Embedding Responsive Graphics and Visualizations in AR through Dynamic Sketching](https://mlsanity.com/paper/2008.08688/) (2020)
- [Human-Robot Interaction in a Shared Augmented Reality Workspace](https://mlsanity.com/paper/2007.12656/) (2020)
- [Communicating Robot Arm Motion Intent Through Mixed Reality Head-mounted Displays](https://mlsanity.com/paper/1708.03655/) (2017)
- [Expressive Robot Motion Timing](https://mlsanity.com/paper/1802.01536/) (2018)
- [Fast Adaptation with Meta-Reinforcement Learning for Trust Modelling in Human-Robot Interaction](https://mlsanity.com/paper/1908.04087/) (2019)
- [ShapeBots: Shape-changing Swarm Robots](https://mlsanity.com/paper/1909.03372/) (2019)
- [Using Socially Expressive Mixed Reality Arms for Enhancing Low-Expressivity Robots](https://mlsanity.com/paper/1911.09713/) (2019)
- [Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D](https://mlsanity.com/paper/2006.09002/) (2020)
- [HapticBots: Distributed Encountered-type Haptics for VR with Multiple Shape-changing Mobile Robots](https://mlsanity.com/paper/2108.10829/) (2021)
- [Augmented-Reality-Based Visualization of Navigation Data of Mobile Robots on the Microsoft Hololens -- Possibilities and Limitations](https://mlsanity.com/paper/1912.12109/) (2019)
- [What the HoloLens Maps Is Your Workspace: Fast Mapping and Set-up of Robot Cells via Head Mounted Displays and Augmented Reality](https://mlsanity.com/paper/2005.12651/) (2020)
- ["Grip-that-there": An Investigation of Explicit and Implicit Task Allocation Techniques for Human-Robot Collaboration](https://mlsanity.com/paper/2102.00581/) (2021)
- [Virtual Barriers in Augmented Reality for Safe and Effective Human-Robot Cooperation in Manufacturing](https://mlsanity.com/paper/2104.05211/) (2021)
- [Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality](https://mlsanity.com/paper/1902.01117/) (2019)
- [PufferBot: Actuated Expandable Structures for Aerial Robots](https://mlsanity.com/paper/2008.07615/) (2020)
- [RoomShift: Room-scale Dynamic Haptics for VR with Furniture-moving Swarm Robots](https://mlsanity.com/paper/2008.08695/) (2020)
- [Perception! Immersion! Empowerment! Superpowers as Inspiration for Visualization](https://mlsanity.com/paper/2108.03524/) (2021)

## Built on by (most cited first, 19 of 19)

- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment](https://mlsanity.com/paper/2405.18525/) (2024)
- [MixerMDM: Learnable Composition of Human Motion Diffusion Models](https://mlsanity.com/paper/2504.01019/) (2025)
- [SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting](https://mlsanity.com/paper/2506.03594/) (2025)
- [AdaptiX -- A Transitional XR Framework for Development and Evaluation of Shared Control Applications in Assistive Robotics](https://mlsanity.com/paper/2310.15887/) (2023)
- [HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agents](https://mlsanity.com/paper/2606.09738/) (2026)
- [Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffusers](https://mlsanity.com/paper/2507.11287/) (2025)
- [The Social Context of Human-Robot Interactions](https://mlsanity.com/paper/2508.13982/) (2025)
- [Need of 6G for the Metaverse Realization](https://mlsanity.com/paper/2301.03386/) (2022)
- [Extended Reality for Enhanced Human-Robot Collaboration: a Human-in-the-Loop Approach](https://mlsanity.com/paper/2403.14597/) (2024)
- [Extended Reality System for Robotic Learning from Human Demonstration](https://mlsanity.com/paper/2409.12862/) (2024)
- [RAMPA: Robotic Augmented Reality for Machine Programming by DemonstrAtion](https://mlsanity.com/paper/2410.13412/) (2024)
- [Unified Geometry and Color Compression Framework for Point Clouds via Generative Diffusion Priors](https://mlsanity.com/paper/2503.18083/) (2025)
- [Diffusion Model-based Activity Completion for AI Motion Capture from Videos](https://mlsanity.com/paper/2505.21566/) (2025)
- [Whom to Respond To? A Transformer-Based Model for Multi-Party Social Robot Interaction](https://mlsanity.com/paper/2507.10960/) (2025)
- [MR-SLAM: Immersive Spatial Supervision for Multi-Robot Mapping via Mixed Reality](https://mlsanity.com/paper/2605.16432/) (2026)
- [fARfetch: Enabling Collocated AR-HRC in Large Visually Diverse Environments with VLM-Driven AR Content Adaptation](https://mlsanity.com/paper/2606.25162/) (2026)
- [Extended Reality as a Mediation Layer for Situated Human Control in Human-Robot Teaming](https://mlsanity.com/paper/2607.25047/) (2026)
- [ORCESTRA: VLM-driven Visual Robot programming in Mixed Reality](https://mlsanity.com/paper/2608.00775/) (2026)

## Abstract (arXiv)

This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers. Augmented and mixed reality (AR/MR) have emerged as a new way to enhance human-robot interaction (HRI) and robotic interfaces (e.g., actuated and shape-changing interfaces). Recently, an increasing number of studies in HCI, HRI, and robotics have demonstrated how AR enables better interactions between people and robots. However, often research remains focused on individual explorations and key design strategies, and research questions are rarely analyzed systematically. In this paper, we synthesize and categorize this research field in the following dimensions: 1) approaches to augmenting reality; 2) characteristics of robots; 3) purposes and benefits; 4) classification of presented information; 5) design components and strategies for visual augmentation; 6) interaction techniques and modalities; 7) application domains; and 8) evaluation strategies. We formulate key challenges and opportunities to guide and inform future research in AR and robotics.