mlsanityData to 7 Oct 2026

DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning

arXiv 2410.24185 ↗First posted 31 Oct 2024cs.RO cs.AI cs.CVcited by 104

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning from human demonstrations is an effective means to teach robots manipulation skills. But data acquisition is a major bottleneck in applying this paradigm more broadly, due to the amount of cost and human effort involved. There has been significant interest in imitation learning for bimanual dexterous robots, like humanoids. Unfortunately, data collection is even more challenging here due to the challenges of simultaneously controlling multiple arms and multi-fingered hands. Automated data generation in simulation is a compelling, scalable alternative to fuel this need for data. To this end, we introduce DexMimicGen, a large-scale automated data generation system that synthesizes trajectories from a handful of human demonstrations for humanoid robots with dexterous hands. We present a collection of simulation environments in the setting of bimanual dexterous manipulation, spanning a range of manipulation behaviors and different requirements for coordination among the two arms. We generate 21K demos across these tasks from just 60 source human demos and study the effect of several data generation and policy learning decisions on agent performance. Finally, we present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task. Generated datasets, simulation environments and additional results are at https://dexmimicgen.github.io/

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 67 of its 81 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

67 of 81 references.

Closest ancestors · 21

Same direction, newest first

  1. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment24 Oct
  2. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulation13 Aug
  3. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning3 Jul
  4. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedback1 Jul
  5. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning2 May
  6. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning27 Feb
  7. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations26 Oct
  8. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  9. 2023Imitating Task and Motion Planning with Visuomotor Transformers25 May
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  11. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning12 Dec
  12. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning20 Oct
9 more
  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  2. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation9 Dec
  3. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation28 Sep
  4. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control28 Feb
  5. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation12 Dec
  6. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperation12 Dec
  7. 2020Visual Imitation Made Easy11 Aug
  8. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  9. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov

Other references · 46

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  8. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  9. 2020Reinforcement Learning with Augmented Datacited by 452
  10. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  11. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  12. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
34 more
  1. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  2. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  3. 2019Learning Latent Plans from Playcited by 310
  4. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  5. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  6. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  7. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  8. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  9. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  10. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  11. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  12. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  13. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  14. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  15. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  16. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 112
  17. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 103
  18. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  19. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  20. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 73
  21. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 71
  22. 2024OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulationcited by 67
  23. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 61
  24. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 49
  25. 2023AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wildcited by 46
  26. 2024RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learningcited by 46
  27. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  28. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  29. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  30. 2024View-Invariant Policy Learning via Zero-Shot Novel View Synthesiscited by 35
  31. 2023Teleoperation of Humanoid Robots: A Surveycited by 32
  32. 2024BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmarkcited by 30
  33. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 27
  34. 2022Data Augmentation for Manipulationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 51; 2026: 52.

104 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  3. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  4. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  5. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  6. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  7. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  8. 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 65
  9. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 57
  10. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  11. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  12. 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
38 more
  1. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  2. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  3. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33
  4. 2025Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentationcited by 31
  5. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  6. 2025RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluationcited by 29
  7. 2025Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstrationcited by 27
  8. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 25
  9. 2025Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimizationcited by 20
  10. 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 17
  11. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 17
  12. 2026Towards Generalizable Robotic Manipulation in Dynamic Environmentscited by 17
  13. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 16
  14. 2025Contrastive Representation Regularization for Vision-Language-Action Modelscited by 16
  15. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 14
  16. 2026Flow Policy Gradients for Robot Controlcited by 14
  17. 2025DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulationcited by 10
  18. 2025TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Modelscited by 10
  19. 2025Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Explorationcited by 9
  20. 2025Generalizable Domain Adaptation for Sim-and-Real Policy Co-Trainingcited by 9
  21. 2026Dexora: Open-source VLA for High-DoF Bimanual Dexteritycited by 9
  22. 2025Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretrainingcited by 8
  23. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 8
  24. 2025Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidancecited by 7
  25. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  26. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
  27. 2026RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learningcited by 7
  28. 2025Constraint-Preserving Data Generation for Visuomotor Policy Learningcited by 6
  29. 2025Learning Affordances at Inference-Time for Vision-Language-Action Modelscited by 6
  30. 2026Motus2: A Self-Evolving General World Model for Dexterous Manipulationcited by 6
  31. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulationcited by 5
  32. 2026You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vectorcited by 5
  33. 2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Controlcited by 5
  34. 2025D-CODA: Diffusion for Coordinated Dual-Arm Data Augmentationcited by 4
  35. 2025R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulationcited by 4
  36. 2025FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generationcited by 4
  37. 2025Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interfacecited by 4
  38. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4

50 most cited of 104 All 104 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning
  2. 2025DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
  3. 2026Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulation ↗
  4. 2025Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey
  5. 2025Dexterous Manipulation through Imitation Learning: A Survey
  6. 2022A System for Imitation Learning of Contact-Rich Bimanual Manipulation Policies
  7. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
  8. 2025DexterityGen: Foundation Controller for Unprecedented Dexterity