# Augmented Reality Demonstrations for Scalable Robot Imitation Learning

arXiv: https://arxiv.org/abs/2403.13910 · HTML: https://mlsanity.com/paper/2403.13910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 21 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 10 of 10)

- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Improving Sample Efficiency in Model-Free Reinforcement Learning from Images](https://mlsanity.com/paper/1910.01741/) (2019)
- [Learning from Suboptimal Demonstration via Self-Supervised Reward Regression](https://mlsanity.com/paper/2010.11723/) (2020)
- [Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation](https://mlsanity.com/paper/1810.07121/) (2018)
- [AR2-D2:Training a Robot Without a Robot](https://mlsanity.com/paper/2306.13818/) (2023)
- [A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies](https://mlsanity.com/paper/2208.00596/) (2022)
- [Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation](https://mlsanity.com/paper/2001.00503/) (2020)
- [Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors](https://mlsanity.com/paper/2210.08121/) (2022)

## Abstract (arXiv)

Robot Imitation Learning (IL) is a widely used method for training robots to perform manipulation tasks that involve mimicking human demonstrations to acquire skills. However, its practicality has been limited due to its requirement that users be trained in operating real robot arms to provide demonstrations. This paper presents an innovative solution: an Augmented Reality (AR)-assisted framework for demonstration collection, empowering non-roboticist users to produce demonstrations for robot IL using devices like the HoloLens 2. Our framework facilitates scalable and diverse demonstration collection for real-world tasks. We validate our approach with experiments on three classical robotics tasks: reach, push, and pick-and-place. The real robot performs each task successfully while replaying demonstrations collected via AR.