mlsanityData to 14 Oct 2026

EgoMimic: Scaling Imitation Learning via Egocentric Video

arXiv 2410.24221 ↗First posted 31 Oct 2024cs.RO cs.CVcited by 175

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

The scale and diversity of demonstration data required for imitation learning is a significant challenge. We present EgoMimic, a full-stack framework which scales manipulation via human embodiment data, specifically egocentric human videos paired with 3D hand tracking. EgoMimic achieves this through: (1) a system to capture human embodiment data using the ergonomic Project Aria glasses, (2) a low-cost bimanual manipulator that minimizes the kinematic gap to human data, (3) cross-domain data alignment techniques, and (4) an imitation learning architecture that co-trains on human and robot data. Compared to prior works that only extract high-level intent from human videos, our approach treats human and robot data equally as embodied demonstration data and learns a unified policy from both data sources. EgoMimic achieves significant improvement on a diverse set of long-horizon, single-arm and bimanual manipulation tasks over state-of-the-art imitation learning methods and enables generalization to entirely new scenes. Finally, we show a favorable scaling trend for EgoMimic, where adding 1 hour of additional hand data is significantly more valuable than 1 hour of additional robot data. Videos and additional information can be found at https://egomimic.github.io/

Key facts. Cited by 175 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 53 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

47 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning26 Aug
  2. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedback1 Jul
  3. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation12 Mar
  4. 2024Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation29 Feb
  5. 2023Any-point Trajectory Modeling for Policy Learning28 Dec
  6. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  8. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play24 Feb
  9. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors20 Oct
  10. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training30 Sep
  11. 2021Manipulator-Independent Representations for Visual Imitation16 Mar
  12. 2020Visual Imitation Made Easy11 Aug
1 more
  1. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations13 Mar

Other references · 34

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  8. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  9. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  10. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  11. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  12. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
22 more
  1. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  2. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  3. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  4. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  5. 2022Human-to-Robot Imitation in the Wildcited by 161
  6. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 148
  7. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
  8. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 121
  9. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 114
  10. 2023On Bringing Robots Homecited by 101
  11. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  12. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  13. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 63
  14. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  15. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 52
  16. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 52
  17. 2023AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wildcited by 47
  18. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  19. 2024Mirage: Cross-Embodiment Zero-Shot Policy Transfer with Cross-Paintingcited by 36
  20. 2024ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperationcited by 34
  21. 2023Polybot: Training One Policy Across Robots While Embracing Variabilitycited by 28
  22. 2023OpenVR: Teleoperation for Manipulationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 58; 2026: 115.

175 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  2. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  3. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 142
  4. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  5. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 128
  6. 2025GR-3 Technical Reportcited by 111
  7. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  8. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
  9. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  10. 2025Humanoid Policy ~ Human Policycited by 78
  11. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  12. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
38 more
  1. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 69
  2. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 66
  3. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
  4. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  5. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 50
  6. 2026World Model for Robot Learning: A Comprehensive Surveycited by 46
  7. 2025H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulationcited by 45
  8. 2025GigaBrain-0: A World Model-Powered Vision-Language-Action Modelcited by 44
  9. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 44
  10. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  11. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 42
  12. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  13. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 38
  14. 2026Native Video-Action Pretraining for Generalizable Robot Controlcited by 36
  15. 2025UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representationscited by 35
  16. 2025EgoZero: Robot Learning from Smart Glassescited by 33
  17. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 32
  18. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  19. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 31
  20. 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 29
  21. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 26
  22. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 22
  23. 2026EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Datacited by 20
  24. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
  25. 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmarkcited by 19
  26. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 19
  27. 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 18
  28. 2025AMPLIFY: Actionless Motion Priors for Robot Learning from Videoscited by 18
  29. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 16
  30. 2025Diffusion Trajectory-guided Policy for Long-horizon Robot Manipulationcited by 15
  31. 2025OPENTOUCH: Bringing Full-Hand Touch to Real-World Interactioncited by 15
  32. 2026Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wildcited by 15
  33. 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Modelscited by 14
  34. 2025Object-centric 3D Motion Field for Robot Learning from Human Videoscited by 14
  35. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  36. 2026ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulationcited by 14
  37. 2025OpenEgo: A Large-Scale Multimodal Egocentric Dataset for Dexterous Manipulationcited by 13
  38. 2025Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videoscited by 13

50 most cited of 175 All 175 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos ↗
  2. 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
  3. 2026Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  4. 2025EgoMI: Learning Active Vision and Whole-Body Manipulation from Egocentric Human Demonstrations
  5. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
  6. 2026EgoSteer: An Open-Source Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos
  7. 2026Wh0: Generative World Models as Scalable Sources of Egocentric Human Hand Manipulation Data
  8. 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Data