mlsanityData to 7 Oct 2026

MimicPlay: Long-Horizon Imitation Learning by Watching Human Play

arXiv 2302.12422 ↗First posted 24 Feb 2023cs.ROcited by 214

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning from human demonstrations is a promising paradigm for teaching robots manipulation skills in the real world. However, learning complex long-horizon tasks often requires an unattainable amount of demonstrations. To reduce the high data requirement, we resort to human play data - video sequences of people freely interacting with the environment using their hands. Even with different morphologies, we hypothesize that human play data contain rich and salient information about physical interactions that can readily facilitate robot policy learning. Motivated by this, we introduce a hierarchical learning framework named MimicPlay that learns latent plans from human play data to guide low-level visuomotor control trained on a small number of teleoperated demonstrations. With systematic evaluations of 14 long-horizon manipulation tasks in the real world, we show that MimicPlay outperforms state-of-the-art imitation learning methods in task success rate, generalization ability, and robustness to disturbances. Code and videos are available at https://mimic-play.github.io

Key facts. Cited by 214 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 46 of its 71 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

46 of 71 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learning19 Sep
  2. 2022Graph Inverse Reinforcement Learning from Diverse Videos28 Jul
  3. 2022Behavior Transformers: Cloning k modes with one stone22 Jun
  4. 2021XIRL: Cross-embodiment Inverse Reinforcement Learning7 Jun
  5. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control28 Feb
  6. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  7. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller21 Nov
  8. 2019Self-Supervised Correspondence in Visuomotor Policy Learning16 Sep
  9. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  9. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  10. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  11. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  12. 2021Implicit Behavioral Cloningcited by 442
25 more
  1. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  2. 2019Learning Latent Plans from Playcited by 310
  3. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  4. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  5. 2022Masked Visual Pre-training for Motor Controlcited by 246
  6. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  7. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  8. 2022Human-to-Robot Imitation in the Wildcited by 157
  9. 2019Temporal Cycle-Consistency Learningcited by 151
  10. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  11. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  12. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  13. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  14. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  15. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  16. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  17. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 80
  18. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 75
  19. 2018TACO: Learning Task Decomposition via Temporal Alignment for Controlcited by 61
  20. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  21. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  22. 2019Graph-Structured Visual Imitationcited by 43
  23. 2019Learning Predictive Models From Observation and Interactioncited by 41
  24. 2022PLATO: Predicting Latent Affordances Through Object-Centric Playcited by 12
  25. 2019Perceptual Values from Observationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 25; 2024: 62; 2025: 80; 2026: 47.

214 papers.

Sort:

By area:

  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  3. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  4. 2024Latent Action Pretraining from Videoscited by 302
  5. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  6. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  7. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  8. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  9. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  10. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  11. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  12. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
38 more
  1. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  2. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  3. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  4. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  5. 2023Large Language Models for Robotics: A Surveycited by 132
  6. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  7. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  8. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  9. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  10. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  11. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  12. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  13. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  14. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 75
  15. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  16. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 69
  17. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  18. 2025Humanoid Policy ~ Human Policycited by 65
  19. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
  20. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  21. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 62
  22. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  23. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  24. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 57
  25. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  26. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  27. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  28. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
  29. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  30. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  31. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  32. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 43
  33. 2024PoCo: Policy Composition from and for Heterogeneous Robot Learningcited by 42
  34. 2024FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Modelcited by 41
  35. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  36. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 38
  37. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  38. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 36

50 most cited of 214 All 214 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Human-to-Robot Interaction: Learning from Video Demonstration for Robot Imitation
  2. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges
  3. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  4. 2025Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space
  5. 2021Playful Interactions for Representation Learning
  6. 2021Imitation Learning: Progress, Taxonomies and Challenges
  7. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation
  8. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration