mlsanityData to 7 Oct 2026

Humanoid Policy ~ Human Policy

arXiv 2503.13441 ↗First posted 17 Mar 2025cs.RO cs.AI cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Training manipulation policies for humanoid robots with diverse data enhances their robustness and generalization across tasks and platforms. However, learning solely from robot demonstrations is labor-intensive, requiring expensive tele-operated data collection which is difficult to scale. This paper investigates a more scalable data source, egocentric human demonstrations, to serve as cross-embodiment training data for robot learning. We mitigate the embodiment gap between humanoids and humans from both the data and modeling perspectives. We collect an egocentric task-oriented dataset (PH2D) that is directly aligned with humanoid manipulation demonstrations. We then train a human-humanoid behavior policy, which we term Human Action Transformer (HAT). The state-action space of HAT is unified for both humans and humanoid robots and can be differentiably retargeted to robot actions. Co-trained with smaller-scale robot data, HAT directly models humanoid robots and humans as different embodiments without additional supervision. We show that human data improves both generalization and robustness of HAT with significantly better data collection efficiency. Code and data: https://human-as-robot.github.io/

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 69 of its 76 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

69 of 76 references.

References · 69

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  6. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  10. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  11. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  12. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
38 more
  1. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  2. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  3. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  4. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  5. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  6. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  7. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  8. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  9. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  10. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  11. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  12. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  13. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  14. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  15. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  16. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  17. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  18. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  19. 2022Human-to-Robot Imitation in the Wildcited by 157
  20. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  21. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  22. 2023Structured World Models from Human Videoscited by 144
  23. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  24. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexteritycited by 128
  25. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  26. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  27. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  28. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  29. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  30. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  31. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  32. 2024UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllerscited by 67
  33. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 62
  34. 2024Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigationcited by 60
  35. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  36. 2025Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learningcited by 57
  37. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  38. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 52

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 24; 2026: 41.

65 papers.

Sort:

By area:

  1. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  2. 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 117
  3. 2025GR-3 Technical Reportcited by 110
  4. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  5. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  6. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Surveycited by 66
  7. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  8. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  9. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  10. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 46
  11. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  12. 2025H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulationcited by 41
38 more
  1. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 34
  2. 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 27
  3. 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videoscited by 27
  4. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 22
  5. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 21
  6. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulationcited by 20
  7. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 17
  8. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 17
  9. 2026EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Datacited by 17
  10. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 14
  11. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 13
  12. 2025OpenEgo: A Large-Scale Multimodal Egocentric Dataset for Dexterous Manipulationcited by 12
  13. 2026ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulationcited by 11
  14. 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrationscited by 11
  15. 2025Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videoscited by 10
  16. 2026SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulationcited by 10
  17. 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulationcited by 10
  18. 2025UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policiescited by 9
  19. 2025METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Modelcited by 9
  20. 2026UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modelingcited by 9
  21. 2025Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretrainingcited by 8
  22. 2025H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videoscited by 8
  23. 2026Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstructioncited by 8
  24. 2025DexMan: Learning Bimanual Dexterous Manipulation from Human and Generated Videoscited by 7
  25. 2026Learning Whole-Body Human-Humanoid Interaction from Human-Human Demonstrationscited by 5
  26. 2025Uni-Hand: Universal Hand Motion Forecasting in Egocentric Viewscited by 4
  27. 2026OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptationcited by 4
  28. 2026ActiveMimic: Egocentric Video Pretraining with Active Perceptioncited by 4
  29. 2026OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulationcited by 4
  30. 2026Data Pyramid for Embodied Manipulation: A Surveycited by 4
  31. 2026AoE: Always-on Egocentric Human Video Collection for Embodied AIcited by 3
  32. 2026FACT: Failure-Aware Causal Training for World-Action Modelscited by 3
  33. 2025TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learningcited by 2
  34. 2025Traj2Action: A Co-Denoising Framework for Trajectory-Guided Human-to-Robot Skill Transfercited by 2
  35. 2026EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videoscited by 2
  36. 2026What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?cited by 2
  37. 2026LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisitioncited by 2
  38. 2026Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videoscited by 1

50 most cited of 65 All 65 (JSON)