# ARCADE: Scalable Demonstration Collection and Generation via Augmented Reality for Imitation Learning

arXiv: https://arxiv.org/abs/2410.15994 · HTML: https://mlsanity.com/paper/2410.15994/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 18 of 18)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study](https://mlsanity.com/paper/1404.3785/) (2014)
- [Learning from Suboptimal Demonstration via Self-Supervised Reward Regression](https://mlsanity.com/paper/2010.11723/) (2020)
- [AR2-D2:Training a Robot Without a Robot](https://mlsanity.com/paper/2306.13818/) (2023)
- [A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies](https://mlsanity.com/paper/2208.00596/) (2022)
- [Back to the Manifold: Recovering from Out-of-Distribution States](https://mlsanity.com/paper/2207.08673/) (2022)
- [Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing](https://mlsanity.com/paper/1703.01270/) (2017)
- [Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation](https://mlsanity.com/paper/2001.00503/) (2020)
- [Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations](https://mlsanity.com/paper/1610.00850/) (2016)
- [One ACT Play: Single Demonstration Behavior Cloning with Action Chunking Transformers](https://mlsanity.com/paper/2309.10175/) (2023)
- [Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Tasks](https://mlsanity.com/paper/2007.11646/) (2020)
- [OpenVR: Teleoperation for Manipulation](https://mlsanity.com/paper/2305.09765/) (2023)
- [Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors](https://mlsanity.com/paper/2210.08121/) (2022)
- [Safe Inverse Reinforcement Learning via Control Barrier Function](https://mlsanity.com/paper/2212.02753/) (2022)

## Built on by (most cited first, 5 of 5)

- [ReBot: Scaling Robot Learning with Real-to-Sim-to-Real Robotic Video Synthesis](https://mlsanity.com/paper/2503.14526/) (2025)
- [BOSS: Benchmark for Observation Space Shift in Long-Horizon Task](https://mlsanity.com/paper/2502.15679/) (2025)
- [Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation](https://mlsanity.com/paper/2605.00244/) (2026)
- [IRIS: An Immersive Robot Interaction System](https://mlsanity.com/paper/2502.03297/) (2025)
- [Sensing Which Modality Matters: Evidence-Gated Regularization for Robust VLA Policies](https://mlsanity.com/paper/2609.03142/) (2026)

## Abstract (arXiv)

Robot Imitation Learning (IL) is a crucial technique in robot learning, where agents learn by mimicking human demonstrations. However, IL encounters scalability challenges stemming from both non-user-friendly demonstration collection methods and the extensive time required to amass a sufficient number of demonstrations for effective training. In response, we introduce the Augmented Reality for Collection and generAtion of DEmonstrations (ARCADE) framework, designed to scale up demonstration collection for robot manipulation tasks. Our framework combines two key capabilities: 1) it leverages AR to make demonstration collection as simple as users performing daily tasks using their hands, and 2) it enables the automatic generation of additional synthetic demonstrations from a single human-derived demonstration, significantly reducing user effort and time. We assess ARCADE's performance on a real Fetch robot across three robotics tasks: 3-Waypoints-Reach, Push, and Pick-And-Place. Using our framework, we were able to rapidly train a policy using vanilla Behavioral Cloning (BC), a classic IL algorithm, which excelled across these three tasks. We also deploy ARCADE on a real household task, Pouring-Water, achieving an 80% success rate.