mlsanityData to 14 Oct 2026

SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment

arXiv 2410.18907 ↗First posted 24 Oct 2024cs.RO cs.AI cs.CVcited by 52

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning from human demonstrations is an effective paradigm for robot manipulation, but acquiring large datasets is costly and resource-intensive, especially for long-horizon tasks. To address this issue, we propose SkillMimicGen (SkillGen), an automated system for generating demonstration datasets from a few human demos. SkillGen segments human demos into manipulation skills, adapts these skills to new contexts, and stitches them together through free-space transit and transfer motion. We also propose a Hybrid Skill Policy (HSP) framework for learning skill initiation, control, and termination components from SkillGen datasets, enabling skills to be sequenced using motion planning at test-time. We demonstrate that SkillGen greatly improves data generation and policy learning performance over a state-of-the-art data generation framework, resulting in the capability to produce data for large scene variations, including clutter, and agents that are on average 24% more successful. We demonstrate the efficacy of SkillGen by generating over 24K demonstrations across 18 task variants in simulation from just 60 human demonstrations, and training proficient, often near-perfect, HSP agents. Finally, we apply SkillGen to 3 real-world manipulation tasks and also demonstrate zero-shot sim-to-real transfer on a long-horizon assembly task. Videos, and more at https://skillgen.github.io.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 75 of its 97 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

75 of 97 references.

Closest ancestors · 23

Same direction, newest first

  1. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning2 May
  2. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  3. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations26 Oct
  4. 2023Human-in-the-Loop Task and Motion Planning for Imitation Learning24 Oct
  5. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  6. 2023Waypoint-Based Imitation Learning for Robotic Manipulation26 Jul
  7. 2023HYDRA: Hybrid Robot Actions for Imitation Learning29 Jun
  8. 2023Imitating Task and Motion Planning with Visuomotor Transformers25 May
  9. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  10. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning12 Dec
  11. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning20 Oct
  12. 2022Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning6 Apr
11 more
  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  2. 2021Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation9 Dec
  3. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replay14 Nov
  4. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation28 Sep
  5. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration13 May
  6. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control28 Feb
  7. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperation12 Dec
  8. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperation12 Dec
  9. 2020Visual Imitation Made Easy11 Aug
  10. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar
  11. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov

Other references · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2023Segment Anythingcited by 8,234
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  10. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  12. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
38 more
  1. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  2. 2020Reinforcement Learning with Augmented Datacited by 468
  3. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  4. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  5. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  7. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  8. 2019Learning Latent Plans from Playcited by 316
  9. 2020Integrated Task and Motion Planningcited by 290
  10. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 289
  11. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  12. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  13. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 225
  14. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  15. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 218
  16. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  17. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169
  18. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 166
  19. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 159
  20. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  21. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
  22. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 104
  23. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  24. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  25. 2022You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstrationcited by 81
  26. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  27. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  28. 2024OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulationcited by 71
  29. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 63
  30. 2020Learning compositional models of robot skills for task and motion planningcited by 48
  31. 2023AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wildcited by 47
  32. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  33. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  34. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  35. 2022Data Augmentation for Manipulationcited by 16
  36. 2021Coarse-to-Fine for Sim-to-Real: Sub-Millimetre Precision Across Wide Task Spacescited by 9
  37. 2022Where To Start? Transferring Simple Skills to Complex Environmentscited by 9
  38. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrationscited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 27; 2026: 24.

52 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  2. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 179
  3. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  4. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
  5. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 54
  6. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 46
  7. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  8. 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 25
  9. 2025Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimizationcited by 21
  10. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 10
  11. 2026Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agentscited by 8
  12. 2025MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulationcited by 7
38 more
  1. 2025Constraint-Preserving Data Generation for Visuomotor Policy Learningcited by 6
  2. 2025R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulationcited by 6
  3. 2025D-CODA: Diffusion for Coordinated Dual-Arm Data Augmentationcited by 5
  4. 2025SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-trainingcited by 5
  5. 2025LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrationscited by 4
  6. 2025FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generationcited by 4
  7. 2025ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learningcited by 4
  8. 2025Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interfacecited by 4
  9. 2026HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planningcited by 4
  10. 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Roboticscited by 4
  11. 2025HybridGen: VLM-Guided Hybrid Planning for Scalable Data Generation of Imitation Learningcited by 3
  12. 2025UMIGen: A Unified Framework for Egocentric Point Cloud Generation and Cross-Embodiment Robotic Imitation Learningcited by 3
  13. 2025Improving Robotic Manipulation Robustness via NICE Scene Surgerycited by 3
  14. 2026FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulationcited by 3
  15. 2025Learning in ImaginationLand: Omnidirectional Policies through 3D Generative Models (OP-Gen)cited by 2
  16. 2025Self-Augmented Robot Trajectory: Efficient Imitation Learning via Safe Self-augmentation with Demonstrator-annotated Precisioncited by 2
  17. 2025DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulationcited by 2
  18. 2026AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Afford Correspondencecited by 2
  19. 2026DREAM: Deployment-Time Demonstration Generation via Real-to-Sim for Scalable Policy Adaptationcited by 2
  20. 2025Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Controlcited by 1
  21. 2025Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogamescited by 1
  22. 2026ExpertGen: Scalable Sim-to-Real Expert Policy Learning from Imperfect Behavior Priorscited by 1
  23. 2026ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curationcited by 1
  24. 2026PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesiscited by 1
  25. 2026SUN: Agentic Robot Policy Learning with Persistent Task Programscited by 1
  26. 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learningcited by 1
  27. 2026DATAFARM: Distribution-Aligned Task and Motion Planning for Fine-Tuning Vision-Language-Action Modelscited by 1
  28. 2026KnowDemo: Knowledge-Guided Robot Demonstration Generation from Human Videoscited by 1
  29. 2025CroSTAta: Cross-State Transition Attention Transformer for Robotic Manipulationcited by 0
  30. 2025Using Temperature Sampling to Effectively Train Robot Learning Policies on Imbalanced Datasetscited by 0
  31. 2026Easy-IIL: Reducing Human Operational Burden in Interactive Imitation Learning via Assistant Expertscited by 0
  32. 2026R2RDreamer: 3D-aware Data Augmentation for Spatially-generalized 2D Manipulation Policiescited by 0
  33. 2026MagicSim: A Unified Infrastructure for Executable Embodied Interactioncited by 0
  34. 2026MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learningcited by 0
  35. 2026ProTracer: Proprioception-Guided Failure Diagnosis in Robot Manipulationcited by 0
  36. 2026TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuningcited by 0
  37. 2026SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptationcited by 0
  38. 2026AeroManip-VLA: Scalable Vision-Language-Action Learning for Aerial Manipulation with RL-Generated Demonstrationscited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models
  2. 2025DynaMimicGen: A Data Generation Framework for Robot Learning of Dynamic Tasks
  3. 2025LLM Trainer: Automated Robotic Data Generation via Demonstration Augmentation using LLMs
  4. 2026SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation
  5. 2024Learning Novel Skills from Language-Generated Demonstrations
  6. 2024VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections
  7. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos
  8. 2026Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation