mlsanityData to 7 Oct 2026

HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos

arXiv 2605.24934 ↗First posted 24 May 2026cs.RO cs.AI cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Human egocentric video captures rich manipulation demonstrations without any robot hardware, yet transferring these skills to robots remains challenging due to the embodiment gap between human and robot in both visual appearance and kinematics. We present HumanEgo, a framework that bridges the embodiment gap by lifting each human demonstration to an entity-level representation of hand-object interaction, and training a flow matching policy with dense auxiliary objectives that amplify supervision from every trajectory. HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable. With only 30 minutes of human videos per task, HumanEgo achieves 92.5% average success across four real-world tasks (75% with just 15 minutes), outperforms matched-time robot teleoperation by 41%, and robustly transfers zero-shot across novel robots, cameras, and environments. We release HumanEgo as an easy-to-use, open-source framework for learning robot policies directly from human data: https://github.com/TX-Leo/HumanEgo

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 70 of its 74 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

70 of 74 references.

References · 70

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
38 more
  1. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  2. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  3. 2020Rescaling Egocentric Visioncited by 529
  4. 2023CoTracker: It is Better to Track Togethercited by 516
  5. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  6. 2023Reconstructing Hands in 3D with Transformerscited by 298
  7. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  8. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  9. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  10. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  11. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  12. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  13. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
  14. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  15. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  16. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  17. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  18. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  19. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  20. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 85
  21. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  22. 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 77
  23. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  24. 2025Humanoid Policy ~ Human Policycited by 65
  25. 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 64
  26. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 62
  27. 2024ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperationcited by 61
  28. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 50
  29. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  30. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  31. 2024Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Trackingcited by 46
  32. 2025Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulationcited by 43
  33. 2025Masquerade: Learning from In-the-wild Human Videos using Data-Editingcited by 38
  34. 2025NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videoscited by 38
  35. 2024SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulationcited by 36
  36. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 36
  37. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  38. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 31

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalizationcited by 8
  2. 2026MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planningcited by 3
  3. 2026LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisitioncited by 2
  4. 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 2
  5. 2026JoyAI-RA 0.5: Scaling Robot Manipulation Learning via Dual Action Alignmentcited by 2
  6. 2026GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulationcited by 2
  7. 2026AffordanceWAM: Affordance-Aware Joint World-Action Modeling for Robot Manipulationcited by 1
  8. 2026Teach and Grow: An Agent-Centered Architecture for General Robot Learningcited by 0
  9. 2026Acting in Meters: Learning Metric Interactions for Precise Robotic Manipulationcited by 0
  10. 2026Skel-WAM: A Hand-Skeleton-Conditioned World Action Model for Human-to-Robot Manipulation Transfercited by 0
  11. 2026Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulationcited by 0
  12. 2026From World Models to World Action Models: Rethinking Next-State Predictioncited by 0
2 more
  1. 2026Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generationcited by 0
  2. 2026TACROSS: An Efficient and Low-Cost Scalable Human Touch System Across Heterogeneous Tactile Sensors for Dexterous Robot Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models
  2. 2026AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization
  3. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
  4. 2026EgoSteer: An Open-Source Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos
  5. 2026Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
  6. 2026EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation
  7. 2026Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  8. 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulation