mlsanityData to 14 Oct 2026

HumanPlus: Humanoid Shadowing and Imitation from Humans

arXiv 2406.10454 ↗First posted 15 Jun 2024cs.RO cs.AI cs.CVcited by 185

Not on the map. Why?

Cited by 39 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

One of the key arguments for building robots that have similar form factors to human beings is that we can leverage the massive human data for training. Yet, doing so has remained challenging in practice due to the complexities in humanoid perception and control, lingering physical gaps between humanoids and humans in morphologies and actuation, and lack of a data pipeline for humanoids to learn autonomous skills from egocentric vision. In this paper, we introduce a full-stack system for humanoids to learn motion and autonomous skills from human data. We first train a low-level policy in simulation via reinforcement learning using existing 40-hour human motion datasets. This policy transfers to the real world and allows humanoid robots to follow human body and hand motion in real time using only a RGB camera, i.e. shadowing. Through shadowing, human operators can teleoperate humanoids to collect whole-body data for learning different tasks in the real world. Using the data collected, we then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously by imitating human skills. We demonstrate the system on our customized 33-DoF 180cm humanoid, autonomously completing tasks such as wearing a shoe to stand up and walk, unloading objects from warehouse racks, folding a sweatshirt, rearranging objects, typing, and greeting another robot with 60-100% success rates using up to 40 demonstrations. Project website: https://humanoid-ai.github.io/

Key facts. Cited by 185 papers in the map (data through 14 Oct 2026). Builds on 55 of its 98 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

55 of 98 references.

References · 55

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  7. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  8. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  9. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  11. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  12. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
38 more
  1. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  2. 2023Reconstructing Hands in 3D with Transformerscited by 306
  3. 2022Masked Visual Pre-training for Motor Controlcited by 248
  4. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  5. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  6. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  7. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  8. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  9. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  10. 2022Human-to-Robot Imitation in the Wildcited by 161
  11. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  12. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  13. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 130
  14. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  15. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  16. 2020Visual Imitation Made Easycited by 111
  17. 2019AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videoscited by 105
  18. 2023On Bringing Robots Homecited by 101
  19. 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
  20. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 96
  21. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  22. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  23. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  24. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  25. 2019Third-Person Visual Imitation Learning via Decoupled Hierarchical Controllercited by 73
  26. 2024Humanoid Locomotion as Next Token Predictioncited by 69
  27. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  28. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 56
  29. 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
  30. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 54
  31. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 46
  32. 2018Feedback Control For Cassie With Deep Reinforcement Learningcited by 45
  33. 2019Graph-Structured Visual Imitationcited by 43
  34. 2023Sim-to-Real Learning for Humanoid Box Loco-Manipulationcited by 36
  35. 2021The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviorscited by 32
  36. 2024Whole-body Humanoid Robot Locomotion with Human Referencecited by 29
  37. 2024Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walkingcited by 21
  38. 2019Whole-Body Geometric Retargeting for Humanoid Robotscited by 20

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 24; 2025: 89; 2026: 72.

185 papers.

Sort:

By area:

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  2. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  3. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 195
  4. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  5. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  6. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  7. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  8. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  9. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
  10. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  11. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 101
  12. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 88
38 more
  1. 2025Humanoid Policy ~ Human Policycited by 78
  2. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 76
  3. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  4. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
  5. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
  6. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  7. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 52
  8. 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 52
  9. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  10. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
  11. 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 46
  12. 2025BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholdscited by 46
  13. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  14. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 44
  15. 2025TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection Systemcited by 44
  16. 2025LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instructioncited by 43
  17. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  18. 2025ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learningcited by 41
  19. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 38
  20. 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Controlcited by 36
  21. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  22. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
  23. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 30
  24. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 29
  25. 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 29
  26. 2025HuB: Learning Extreme Humanoid Balancecited by 28
  27. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 27
  28. 2025Learning Getting-Up Policies for Real-World Humanoid Robotscited by 26
  29. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 25
  30. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 22
  31. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 22
  32. 2025Reconstructing Humans with a Biomechanically Accurate Skeletoncited by 22
  33. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  34. 2025Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimizationcited by 21
  35. 2025Learning coordinated badminton skills for legged manipulatorscited by 19
  36. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
  37. 2026Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matchingcited by 19
  38. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18

50 most cited of 185 All 185 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining ↗
  2. 2026EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration
  3. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos
  4. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
  5. 2026Psi0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation
  6. 2026EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Models
  7. 2025EgoMI: Learning Active Vision and Whole-Body Manipulation from Egocentric Human Demonstrations
  8. 2026Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments