mlsanityData to 7 Oct 2026

MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

arXiv 2310.17596 ↗First posted 26 Oct 2023cs.RO cs.AI cs.CVcited by 308

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents. However, the demonstrations can be extremely costly and time-consuming to collect. We introduce MimicGen, a system for automatically synthesizing large-scale, rich datasets from only a small number of human demonstrations by adapting them to new contexts. We use MimicGen to generate over 50K demonstrations across 18 tasks with diverse scene configurations, object instances, and robot arms from just ~200 human demonstrations. We show that robot agents can be effectively trained on this generated dataset by imitation learning to achieve strong performance in long-horizon and high-precision tasks, such as multi-part assembly and coffee preparation, across broad initial state distributions. We further demonstrate that the effectiveness and utility of MimicGen data compare favorably to collecting additional human demonstrations, making it a powerful and economical approach towards scaling up robot learning. Datasets, simulation environments, videos, and more at https://mimicgen.github.io .

Key facts. Cited by 308 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 94 of its 112 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

94 of 112 references.

Closest ancestors · 19

Same direction, newest first

  1. 2023Imitating Task and Motion Planning with Visuomotor Transformers25 May
  2. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning12 Dec
  3. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning20 Oct
  4. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors20 Oct
  5. 2022Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning6 Apr
  6. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  7. 2022Practical Imitation Learning in the Real World via Task Consistency Loss3 Feb
  8. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation9 Dec
  9. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replay14 Nov
  10. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation28 Sep
  11. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration13 May
  12. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learning31 Mar
7 more
  1. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control28 Feb
  2. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation12 Dec
  3. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperation12 Dec
  4. 2020Visual Imitation Made Easy11 Aug
  5. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  6. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov
  7. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 75

  1. 2017Mask R-CNNcited by 7,259
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  12. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
38 more
  1. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  2. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  4. 2020Reinforcement Learning with Augmented Datacited by 452
  5. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  7. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  8. 2019Learning Latent Plans from Playcited by 310
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  10. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  11. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  12. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  13. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  14. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  15. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  16. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  17. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  18. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  19. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  20. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  21. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  22. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  23. 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 155
  24. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  25. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  26. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 131
  27. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  28. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  29. 2022Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Imagescited by 120
  30. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 103
  31. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  32. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  33. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  34. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  35. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  36. 2021BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Modelscited by 87
  37. 2019IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Datacited by 85
  38. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 48; 2025: 112; 2026: 147.

308 papers.

Sort:

By area:

  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  4. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  5. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  6. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  7. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  8. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  9. 2024THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulationcited by 128
  10. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  11. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  12. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
38 more
  1. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  2. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  3. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  4. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  5. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 68
  6. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  7. 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 65
  8. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
  9. 2024Equivariant Diffusion Policycited by 61
  10. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
  11. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 57
  12. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  13. 2024GRUtopia: Dream General Robots in a City at Scalecited by 54
  14. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  15. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 50
  16. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  17. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  18. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 46
  19. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  20. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  21. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  22. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  23. 2024A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLMcited by 44
  24. 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
  25. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  26. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  27. 2024Instant Policy: In-Context Imitation Learning via Graph Diffusioncited by 38
  28. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  29. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  30. 2024View-Invariant Policy Learning via Zero-Shot Novel View Synthesiscited by 35
  31. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 34
  32. 2024Multi-Task Interactive Robot Fleet Learning with Visual World Modelscited by 33
  33. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33
  34. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  35. 2025EXPO: Stable Reinforcement Learning with Expressive Policiescited by 30
  36. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  37. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  38. 2025Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactionscited by 27

50 most cited of 308 All 308 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models
  2. 2025LLM Trainer: Automated Robotic Data Generation via Demonstration Augmentation using LLMs
  3. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvement
  4. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  5. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play
  6. 2025DIRIGENt: End-To-End Robotic Imitation of Human Demonstrations Based on a Diffusion Model
  7. 2021Imitation Learning: Progress, Taxonomies and Challenges
  8. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations