Mirror Learning
On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning
Abstract (arXiv)
We investigate imitation learning through the lens of third-person observation and propose a framework for mirror learning: acquiring actionable policies from passive observation. While behavior cloning (BC) excels under dense, well-aligned first-person data, it fundamentally fails to leverage the rich observational signals arising from third-person demonstrations that humans and animals routinely exploit. We introduce a method that composes (i) a learned perspective transformation that places learners in demonstrators' shoes using a fine-tuned video diffusion model and (ii) an inverse dynamics model that infers action trajectories in the learners' control space. This enables the synthesis of mirror data, pseudo first-person expert data generated from third-person observations of demonstrator behavior. Empirically, we show that mirror data alone can train effective policies, and that augmenting first-person BC training with mirror data further improves downstream policy performance. Our results suggest that modern generative world models implicitly encode sufficient structure to enable a scalable and safe alternative to teleoperation-heavy data collection.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 44 of its 58 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
44 of 58 references.
Closest ancestors · 1
Same direction, newest first
Other references · 43
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2022Video Diffusion Modelscited by 2,305
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2016End to End Learning for Self-Driving Carscited by 1,009
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2015End-to-End Training of Deep Visuomotor Policiescited by 875
31 more
- 2018Behavioral Cloning from Observationcited by 492
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2022Flexible Diffusion Modeling of Long Videoscited by 294
- 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
- 2025World Simulation with Video Foundation Models for Physical AIcited by 200
- 2023COLMAP-Free 3D Gaussian Splattingcited by 193
- 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
- 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
- 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
- 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 154
- 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
- 2022Learning from All Vehiclescited by 135
- 2017Third-Person Imitation Learningcited by 103
- 2025DiffuEraser: A Diffusion Model for Video Inpaintingcited by 72
- 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 62
- 2024A Survey for Foundation Models in Autonomous Drivingcited by 52
- 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 41
- 2021Learning by Watchingcited by 22
- 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
- 2025From Image to Video: An Empirical Study of Diffusion Representationscited by 15
- 2025Exo2EgoSyn: Unlocking Foundation Video Generation Models for Exocentric-to-Egocentric Video Synthesiscited by 15
- 2024ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Modelscited by 12
- 2025PLAICraft: Large-Scale Time-Aligned Vision-Speech-Action Dataset for Embodied AIcited by 10
- 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
- 2024Semantically Consistent Video Inpainting with Conditional Diffusion Modelscited by 8
- 2025EgoX: Egocentric Video Generation from a Single Exocentric Videocited by 8
- 2022Conditional Permutation Invariant Flowscited by 6
- 2025Control-ITRA: Controlling the Behavior of a Driving Modelcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021Manipulator-Independent Representations for Visual Imitation
- 2026Imitation from Heterogeneous Demonstrations using Grounded Latent-Action World Models
- 2026MirrorDuo: Reflection-Consistent Visuomotor Learning from Mirrored Demonstration Pairs ↗
- 2024A Dual Approach to Imitation Learning from Observations with Offline Datasets
- 2025Learning a Thousand Tasks in a Day
- 2022Diverse Imitation Learning via Self-Organizing Generative Models
- 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
- 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play