# Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation

arXiv: https://arxiv.org/abs/2605.00244 · HTML: https://mlsanity.com/paper/2605.00244/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation](https://mlsanity.com/paper/2311.01455/) (2023)
- [DreamGen: Unlocking Generalization in Robot Learning through Video World Models](https://mlsanity.com/paper/2505.12705/) (2025)
- [Scaling Robot Learning with Semantically Imagined Experience](https://mlsanity.com/paper/2302.11550/) (2023)
- [Open-TeleVision: Teleoperation with Immersive Active Visual Feedback](https://mlsanity.com/paper/2407.01512/) (2024)
- [Expressive Whole-Body Control for Humanoid Robots](https://mlsanity.com/paper/2402.16796/) (2024)
- [URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images](https://mlsanity.com/paper/2405.11656/) (2024)
- [GenAug: Retargeting behaviors to unseen situations via Generative Augmentation](https://mlsanity.com/paper/2302.06671/) (2023)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning](https://mlsanity.com/paper/2212.05711/) (2022)
- [OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation](https://mlsanity.com/paper/2403.07870/) (2024)
- [Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models](https://mlsanity.com/paper/2310.18308/) (2023)
- [ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedback](https://mlsanity.com/paper/2410.08464/) (2024)
- [AR2-D2:Training a Robot Without a Robot](https://mlsanity.com/paper/2306.13818/) (2023)
- [Compositional Sculpting of Iterative Generative Processes](https://mlsanity.com/paper/2309.16115/) (2023)
- [Learning Visual Parkour from Generated Images](https://mlsanity.com/paper/2411.00083/) (2024)
- [DexHub and DART: Towards Internet Scale Robot Data Collection](https://mlsanity.com/paper/2411.02214/) (2024)
- [ARCADE: Scalable Demonstration Collection and Generation via Augmented Reality for Imitation Learning](https://mlsanity.com/paper/2410.15994/) (2024)
- [EVE: Enabling Anyone to Train Robots using Augmented Reality](https://mlsanity.com/paper/2404.06089/) (2024)
- [Puppeteer Your Robot: Augmented Reality Leader-Follower Teleoperation](https://mlsanity.com/paper/2407.11741/) (2024)

## Built on by (most cited first, 4 of 4)

- [EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics](https://mlsanity.com/paper/2609.27308/) (2026)
- [EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devices](https://mlsanity.com/paper/2605.16797/) (2026)
- [NavGen: Visual Generative Models as a Scalable Data Engine for Embodied 3D Navigation](https://mlsanity.com/paper/2609.30770/) (2026)
- [Pre-training Visual Dexterity in Simulation](https://mlsanity.com/paper/2608.15917/) (2026)

## Abstract (arXiv)

We introduce Lucid-XR, a generative data engine for creating diverse and realistic-looking multi-modal data to train real-world robotic systems. At the core of Lucid-XR is vuer, a web-based physics simulation environment that runs directly on the XR headset, enabling internet-scale access to immersive, latency-free virtual interactions without requiring specialized equipment. The complete system integrates on-device physics simulation with human-to-robot pose retargeting. Data collected is further amplified by a physics-guided video generation pipeline steerable via natural language specifications. We demonstrate zero-shot transfer of robot visual policies to unseen, cluttered, and badly lit evaluation environments, after training entirely on Lucid-XR's synthetic data. We include examples across dexterous manipulation tasks that involve soft materials, loosely bound particles, and rigid body contact. Project website: https://lucidxr.github.io