mlsanityData to 7 Oct 2026

EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World

arXiv 2604.07607 ↗First posted 8 Apr 2026cs.RO cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

Robot learning increasingly depends on large and diverse data, yet robot data collection remains expensive and difficult to scale. Egocentric human data offer a promising alternative by capturing rich manipulation behavior across everyday environments. However, existing human datasets are often limited in scope, difficult to extend, and fragmented across institutions. We introduce EgoVerse, a collaborative platform for human data-driven robot learning that unifies data collection, processing, and access under a shared framework, enabling contributions from individual researchers, academic labs, and industry partners. The current release includes 1,362 hours (80k episodes) of human demonstrations spanning 1,965 tasks, 240 scenes, and 2,087 unique demonstrators, with standardized formats, manipulation-relevant annotations, and tooling for downstream learning. Beyond the dataset, we conduct a large-scale study of human-to-robot transfer with experiments replicated across multiple labs, tasks, and robot embodiments under shared protocols. We find that policy performance generally improves with increased human data, but that effective scaling depends on alignment between human data and robot learning objectives. Together, the dataset, platform, and study establish a foundation for reproducible progress in human data-driven robot learning. Videos and additional information can be found at https://egoverse.ai/

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 49 of its 54 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

49 of 54 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  6. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  8. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  9. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  10. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  11. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  12. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
37 more
  1. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  2. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  3. 2024Latent Action Pretraining from Videoscited by 302
  4. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  5. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  6. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  7. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 180
  8. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  9. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  10. 2022Human-to-Robot Imitation in the Wildcited by 157
  11. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  12. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  13. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  14. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  15. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  16. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  17. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  18. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  19. 2025Humanoid Policy ~ Human Policycited by 65
  20. 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 64
  21. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 57
  22. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
  23. 2024Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Trackingcited by 46
  24. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  25. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 36
  26. 2025Vision in Action: Learning Active Perception from Human Demonstrationscited by 33
  27. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 31
  28. 2025EgoZero: Robot Learning from Smart Glassescited by 28
  29. 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 27
  30. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 21
  31. 2025What Matters in Learning from Large-Scale Datasets for Robot Manipulationcited by 20
  32. 2025EMMA: Scaling Mobile Manipulation via Egocentric Human Datacited by 20
  33. 2025ImMimic: Cross-Domain Imitation from Human Videos via Mapping and Interpolationcited by 19
  34. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 14
  35. 2025Scaling Cross-Embodiment World Models for Dexterous Manipulationcited by 13
  36. 2025Guiding Data Collection via Factored Scaling Curvescited by 10
  37. 2025OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfercited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 49.

49 papers.

Sort:

By area:

  1. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  2. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  3. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  4. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  5. 2026EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Datacited by 17
  6. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 14
  7. 2026World Action Models: A Surveycited by 10
  8. 2026HumanNet: Scaling Human-centric Video Learning to One Million Hourscited by 9
  9. 2026Do as I Do: Dexterous Manipulation Data from Everyday Human Videoscited by 9
  10. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  11. 2026HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editingcited by 4
  12. 2026Causally Debiased Latent Action Model for Embodied Action Conditioned World Modelscited by 3
37 more
  1. 2026MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planningcited by 3
  2. 2026StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Videocited by 2
  3. 2026YUBI: Yielding Universal Bidigital Interface for Bimanual Dexterous Manipulation at Scalecited by 2
  4. 2026HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretrainingcited by 2
  5. 2026SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulationcited by 2
  6. 2026JoyAI-RA 0.5: Scaling Robot Manipulation Learning via Dual Action Alignmentcited by 2
  7. 2026LD4WAM: Learning Latent Dynamics from Human Videos for World Action Modelscited by 2
  8. 2026How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learningcited by 1
  9. 2026Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Controlcited by 1
  10. 2026Learning Action Priors for Cross-embodiment Robot Manipulationcited by 1
  11. 2026Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Modelcited by 1
  12. 2026OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policiescited by 1
  13. 2026MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligencecited by 1
  14. 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Datacited by 1
  15. 2026Long-WAM: Scaling the Context of World-Action Modelscited by 1
  16. 2026EgoKit: Towards Unified Low-Cost Egocentric Data Collection with Heterogeneous Devicescited by 0
  17. 2026EgoExoMoCap: Distributed Ego-Exo Human Motion Capturecited by 0
  18. 2026ModPack: Extensible Teleoperation Interface for Bimanual Mobile Manipulationcited by 0
  19. 2026EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstrationcited by 0
  20. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0
  21. 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulationcited by 0
  22. 2026N0-Foundation: Towards the Age of Tactile Intelligencecited by 0
  23. 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learningcited by 0
  24. 2026DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imaginationcited by 0
  25. 2026HuRo: Robotizing Human Videos for Scalable VLA Pretrainingcited by 0
  26. 2026ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample Weightingcited by 0
  27. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
  28. 2026GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videoscited by 0
  29. 2026ActiveScale: Scaling Active Perception for Robots across Model, Data, and Hardwarecited by 0
  30. 2026AtomEgo: Exploring Ego-Robot Integration for Embodied Foundation Model Pretrainingcited by 0
  31. 2026CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Modelcited by 0
  32. 2026TANDEM: Task and Motion Planning with As-Needed Demonstrations for Efficient Vision-Language-Action Model Fine-tuningcited by 0
  33. 2026EgoSpeedUp: Transferring Human Manipulation Tempo to Robot Policiescited by 0
  34. 2026From World Models to World Action Models: Rethinking Next-State Predictioncited by 0
  35. 2026InfiniHand: Streaming World-Space Hand Motion Estimation from Egocentric Videocited by 0
  36. 2026Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?cited by 0
  37. 2026What 30,000 Hours of Ego-centric Video Does Not Teachcited by 0