mlsanityData to 14 Oct 2026

EgoZero: Robot Learning from Smart Glasses

arXiv 2505.20290 ↗First posted 26 May 2025cs.RO cs.AIcited by 33

Not on the map. Why?

Abstract (arXiv)

Despite recent progress in general purpose robotics, robot policies still lag far behind basic human capabilities in the real world. Humans interact constantly with the physical world, yet this rich data resource remains largely untapped in robot learning. We propose EgoZero, a minimal system that learns robust manipulation policies from human demonstrations captured with Project Aria smart glasses, and zero robot data. EgoZero enables: (1) extraction of complete, robot-executable actions from in-the-wild, egocentric, human demonstrations, (2) compression of human visual observations into morphology-agnostic state representations, and (3) closed-loop policy learning that generalizes morphologically, spatially, and semantically. We deploy EgoZero policies on a gripper Franka Panda robot and demonstrate zero-shot transfer with 70% success rate over 7 manipulation tasks and only 20 minutes of data collection per task. Our results suggest that in-the-wild human data can serve as a scalable foundation for real-world robot learning - paving the way toward a future of abundant, diverse, and naturalistic training data for robots. Code and videos are available at https://egozero-robot.github.io.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 44 of its 59 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

44 of 59 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  10. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  11. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
32 more
  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  2. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  3. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  6. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  7. 2023Emergent Correspondence from Image Diffusioncited by 446
  8. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  9. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  10. 2023Reconstructing Hands in 3D with Transformerscited by 306
  11. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  12. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  13. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  14. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  15. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  16. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
  17. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  18. 2022Human-to-Robot Imitation in the Wildcited by 161
  19. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  20. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 134
  21. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  22. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  23. 2024BAKU: An Efficient Transformer for Multi-Task Policy Learningcited by 80
  24. 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 70
  25. 2025Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulationcited by 47
  26. 2024Hand-Object Interaction Pretraining from Videoscited by 39
  27. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  28. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 32
  29. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 28
  30. 2024P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policiescited by 16
  31. 2021Semantic constraints to represent common sense required in household actions for multi-modal Learning-from-observation robotcited by 5
  32. 2023Designing Anthropomorphic Soft Hands through Interactioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 19.

33 papers.

Sort:

By area:

  1. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  3. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
  4. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 42
  5. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 32
  6. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 19
  7. 2025AMPLIFY: Actionless Motion Priors for Robot Learning from Videoscited by 18
  8. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 16
  9. 2025Object-centric 3D Motion Field for Robot Learning from Human Videoscited by 14
  10. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  11. 2026ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulationcited by 14
  12. 2026Do as I Do: Dexterous Manipulation Data from Everyday Human Videoscited by 13
21 more
  1. 2025X-Humanoid: Robotize Human Videos to Generate Humanoid Videos at Scalecited by 12
  2. 2025Developing Vision-Language-Action Model from Egocentric Videoscited by 11
  3. 2025OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfercited by 10
  4. 2026ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretrainingcited by 10
  5. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrationscited by 6
  6. 2026Robot Learning from Human Videos: A Surveycited by 5
  7. 2026Human Universal Graspingcited by 5
  8. 2026HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editingcited by 5
  9. 2025EmbodiSwap for Zero-Shot Robot Imitation Learningcited by 2
  10. 2026EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videoscited by 2
  11. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 2
  12. 2025EgoLoc: A Generalizable Solution for Temporal Interaction Localization in Egocentric Videoscited by 1
  13. 2025Scene Text Detection and Recognition "in light of" Challenging Environmental Conditions using Aria Glasses Egocentric Vision Camerascited by 0
  14. 2026MotionBits: Video Segmentation through Motion-Level Analysis of Rigid Bodiescited by 0
  15. 2026Human2Any: Human-to-Robot Transfer via Constraint-Aware Compositional Planningcited by 0
  16. 2026EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstrationcited by 0
  17. 2026AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learningcited by 0
  18. 2026From Seeing to Acting: Smart Glasses as First-Person Intelligence Platformscited by 0
  19. 2026RoboCap: A New Platform for Egocentric Robot Learningcited by 0
  20. 2026RLHND: Video Foundation Models as Physically Grounded Hand Trackers for Robot Learningcited by 0
  21. 2026Steertuning Policies from a Few Human Videoscited by 0