HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos
Not on the map. Why?
Abstract (arXiv)
Enabling humanoid robots to perform agile and adaptive interactions has long been a core challenge in robotics. Current approaches are bottlenecked by either the scarcity of realistic interaction data or the need for meticulous, task-specific reward engineering, which limits their scalability. To narrow this gap, we present HumanX, a unified framework that compiles human video into generalizable, real-world interaction skills for humanoids, without per-task design. HumanX integrates two co-designed components: XGen, an anchor-centric data generation pipeline that yields physically plausible robot interaction data from monocular video and augments it to create a diverse, scalable training dataset; and XMimic, an interaction-prioritized learning framework to achieve generalizable interaction skills from seconds of demonstration. Evaluated across five distinct domains--basketball, football, badminton, cargo pickup, and reactive fighting--HumanX successfully acquires 9 different skills and transfers them zero-shot to a physical Unitree G1 humanoid. The learned capabilities include complex maneuvers such as pump-fake turnaround fadeaway jumpshots without any external perception, as well as interactive tasks like sustained human-robot passing sequences over 10 consecutive cycles--learned from a single video demonstration. Our experiments show that HumanX achieves higher generalization success than prior methods, demonstrating a scalable pathway within a class of anchorable interactions for learning versatile, real-world robot interactive skills.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
56 of 66 references.
References · 56
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
- 2019Learning joint reconstruction of hands and manipulated objectscited by 378
- 2025SAM 3D: 3Dfy Anything in Imagescited by 247
- 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
- 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
- 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
- 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 165
- 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
38 more
- 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 163
- 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 157
- 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
- 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
- 2022DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulationcited by 137
- 2024World-Grounded Human Motion Recovery via Gravity-View Coordinatescited by 137
- 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
- 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 130
- 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
- 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 101
- 2024Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Controlcited by 99
- 2024Humanoid Parkour Learningcited by 96
- 2025GMT: General Motion Tracking for Humanoid Whole-Body Controlcited by 83
- 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
- 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 76
- 2025Retargeting Matters: General Motion Retargeting for Humanoid Motion Trackingcited by 75
- 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
- 2021Synthesizing Diverse and Physically Stable Grasps with Arbitrary Hand Structures using Differentiable Force Closure Estimatorcited by 71
- 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
- 2025Visual Imitation Enables Contextual Humanoid Controlcited by 68
- 2024WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contactscited by 61
- 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 52
- 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
- 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 46
- 2025BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholdscited by 46
- 2025CLONE: Closed-Loop Whole-Body Humanoid Teleoperation for Long-Horizon Taskscited by 38
- 2025HDMI: Learning Interactive Humanoid Whole-Body Control from Human Videoscited by 38
- 2025UniTracker: Learning Universal Whole-Body Motion Tracker for Humanoid Robotscited by 37
- 2025Track Any Motions under Any Disturbancescited by 35
- 2025HITTER: A HumanoId Table TEnnis Robot via Hierarchical Planning and Learningcited by 33
- 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
- 2023Composite Motion Learning with Task Controlcited by 28
- 2025Learning Getting-Up Policies for Real-World Humanoid Robotscited by 26
- 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 25
- 2025LangWBC: Language-directed Humanoid Whole-Body Control via End-to-end Learningcited by 22
- 2025Learning coordinated badminton skills for legged manipulatorscited by 19
- 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
- 2025Behavior Foundation Model for Humanoid Robotscited by 15
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026World-Action Models for Robot Learning and Control: A Surveycited by 6
- 2026SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Frameworkcited by 3
- 2026Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priorscited by 2
- 2026Scaling Behavior Foundation Model for Humanoid Robotscited by 2
- 2026Switch: Learning Agile Skills Switching for Humanoid Robotscited by 1
- 2026RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learningcited by 1
- 2026TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutescited by 0
- 2026Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Trackingcited by 0
- 2026HIGenNTO: Scalable Humanoid Interaction Generation via Noise-Space Trajectory Optimizationcited by 0
- 2026Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulationcited by 0
- 2026Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Librarycited by 0
- 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulationcited by 0
- 2026From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion10 Oct 2026
- 2026CoDance: Learning Reactive and Compliant Human-Humanoid Interaction from Video4 Oct 2026
- 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation1 Oct 2026
- 2026Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Library30 Sep 2026
- 2026Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation29 Sep 2026
- 2026HIGenNTO: Scalable Humanoid Interaction Generation via Noise-Space Trajectory Optimization18 Sep 2026
- 2026World-Action Models for Robot Learning and Control: A Survey13 Sep 2026
- 2026Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking20 Aug 2026
- 2026Scaling Behavior Foundation Model for Humanoid Robots16 Jul 2026
- 2026TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes24 Jun 2026
- 2026RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning9 Jun 2026
- 2026Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors21 May 2026
- 2026Switch: Learning Agile Skills Switching for Humanoid Robotscited by 1
- 2026RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learningcited by 1
- 2026Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Librarycited by 0
- 2026From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motioncited by 0
- 2026From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion10 Oct 2026
- 2026Locomotion-Grounded Humanoid Soccer: Task-Gated Reinforcement Learning of a Multi-Directional Kicking Library30 Sep 2026
- 2026RoboNaldo: Accurate, Stable and Powerful Humanoid Soccer Shooting via Motion-Guided Curriculum Reinforcement Learning9 Jun 2026
- 2026Switch: Learning Agile Skills Switching for Humanoid Robots16 Apr 2026
- 2026World-Action Models for Robot Learning and Control: A Survey13 Sep 2026
- 2026SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Frameworkcited by 3
- 2026Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priorscited by 2
- 2026Scaling Behavior Foundation Model for Humanoid Robotscited by 2
- 2026TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutescited by 0
- 2026Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Trackingcited by 0
- 2026HIGenNTO: Scalable Humanoid Interaction Generation via Noise-Space Trajectory Optimizationcited by 0
- 2026Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulationcited by 0
- 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulationcited by 0
- 2026CoDance: Learning Reactive and Compliant Human-Humanoid Interaction from Videocited by 0
- 2026CoDance: Learning Reactive and Compliant Human-Humanoid Interaction from Video4 Oct 2026
- 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation1 Oct 2026
- 2026Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation29 Sep 2026
- 2026HIGenNTO: Scalable Humanoid Interaction Generation via Noise-Space Trajectory Optimization18 Sep 2026
- 2026Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking20 Aug 2026
- 2026Scaling Behavior Foundation Model for Humanoid Robots16 Jul 2026
- 2026TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes24 Jun 2026
- 2026Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors21 May 2026
- 2026SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework19 May 2026
Similar, not cited
Close in wording, no citation either way
- 2024Learning from Massive Human Videos for Universal Humanoid Pose Control
- 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimicking
- 2026InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skills
- 2026RoboTacDex: A Dexterous Visual-Tactile-Action Dataset for Humanoid Manipulation ↗
- 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
- 2026Learning Human-Like Badminton Skills for Humanoid Robots
- 2026Being-M0.7: A Latent World-Action Model for Humanoid Robots
- 2026Psi0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation