mlsanityData to 7 Oct 2026

Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control

arXiv 2412.07773 ↗First posted 10 Dec 2024cs.RO cs.AI cs.LGcited by 41

Not on the map. Why?

Abstract (arXiv)

Humanoid robots require both robust lower-body locomotion and precise upper-body manipulation. While recent Reinforcement Learning (RL) approaches provide whole-body loco-manipulation policies, they lack precise manipulation with high DoF arms. In this paper, we propose decoupling upper-body control from locomotion, using inverse kinematics (IK) and motion retargeting for precise manipulation, while RL focuses on robust lower-body locomotion. We introduce PMP (Predictive Motion Priors), trained with Conditional Variational Autoencoder (CVAE) to effectively represent upper-body motions. The locomotion policy is trained conditioned on this upper-body motion representation, ensuring that the system remains robust with both manipulation and locomotion. We show that CVAE features are crucial for stability and robustness, and significantly outperforms RL-based whole-body control in precise manipulation. With precise upper-body motion and robust lower-body locomotion control, operators can remotely control the humanoid to walk around and explore different environments, while performing diverse manipulation tasks.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 10 Dec 2024.

Builds on

33 of 49 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  3. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  4. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  5. 2021Character Controllers Using Motion VAEscited by 182
  6. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  7. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  8. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  9. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  10. 2023Extreme Parkour with Legged Robotscited by 120
  11. 2024Expressive Whole-Body Control for Humanoid Robotscited by 117
  12. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
21 more
  1. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 112
  2. 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 109
  3. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  4. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
  5. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 91
  6. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 73
  7. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 68
  8. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 67
  9. 2024UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllerscited by 67
  10. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  11. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 49
  12. 2024WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contactscited by 49
  13. 2023DribbleBot: Dynamic Legged Manipulation in the Wildcited by 33
  14. 2022Combining Learning-based Locomotion Policy with Model-based Manipulation for Legged Mobile Manipulatorscited by 32
  15. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 30
  16. 2023Sim-to-Real Learning for Humanoid Box Loco-Manipulationcited by 30
  17. 2023Learning Whole-body Manipulation for Quadrupedal Robotcited by 20
  18. 2024Whole-body Humanoid Robot Locomotion with Human Referencecited by 18
  19. 2024Strategy and Skill Learning for Physics-based Table Tennis Animationcited by 17
  20. 2022Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racingcited by 13
  21. 2024LocoMan: Advancing Versatile Quadrupedal Dexterity with Lightweight Loco-Manipulatorscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 27; 2026: 14.

41 papers.

Sort:

By area:

  1. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  2. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 86
  3. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  4. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 65
  5. 2025Humanoid Policy ~ Human Policycited by 65
  6. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
  7. 2025Humanoid-VLA: Towards Universal Humanoid Control with Visual Integrationcited by 43
  8. 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 39
  9. 2025TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection Systemcited by 39
  10. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 22
  11. 2025Learning Getting-Up Policies for Real-World Humanoid Robotscited by 21
  12. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 21
29 more
  1. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 17
  2. 2026TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Controlcited by 14
  3. 2025A Unified and General Humanoid Whole-Body Controller for Versatile Locomotioncited by 10
  4. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 10
  5. 2026HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulationcited by 10
  6. 2025Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretrainingcited by 8
  7. 2026SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulationcited by 8
  8. 2025Towards Adaptable Humanoid Control via Adaptive Motion Trackingcited by 6
  9. 2026SaPaVe: Towards Active Perception and Manipulation in Vision-Language-Action Models for Roboticscited by 6
  10. 2025Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learningcited by 5
  11. 2025Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robotscited by 5
  12. 2025Natural Humanoid Robot Locomotion with Generative Motion Priorcited by 4
  13. 2025Coordinated Humanoid Manipulation with Choice Policiescited by 4
  14. 2026CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperationcited by 4
  15. 2026OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulationcited by 4
  16. 2025Towards Synthesized and Editable Motion In-Betweening Through Part-Wise Phase Representationcited by 3
  17. 2025Task Tokens: A Flexible Approach to Adapting Behavior Foundation Modelscited by 3
  18. 2026InterReal: A Unified Physics-Based Imitation Framework for Learning Human-Object Interaction Skillscited by 3
  19. 2026PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generationcited by 3
  20. 2025FLAM: Foundation Model-Based Body Stabilization for Humanoid Locomotion and Manipulationcited by 2
  21. 2025IKDiffuser: a Diffusion-based Generative Inverse Kinematics Solver for Kinematic Treescited by 2
  22. 2025TrajBooster: Boosting Humanoid Whole-Body Manipulation via Trajectory-Centric Learningcited by 2
  23. 2026REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learningcited by 2
  24. 2025MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotioncited by 1
  25. 2026HERO: Learning Humanoid End-Effector Control for Visual Whole-Body Open-Vocabulary Object Graspingcited by 0
  26. 2026LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting Worldcited by 0
  27. 2026AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidancecited by 0
  28. 2026ModPack: Extensible Teleoperation Interface for Bimanual Mobile Manipulationcited by 0
  29. 2026Towards Miniature Humanoid Tele-Loco-Manipulation Using Virtual Reality and Reinforcement Learningcited by 0