mlsanityData to 7 Oct 2026

Hierarchical World Models as Visual Whole-Body Humanoid Controllers

arXiv 2405.18418 ↗First posted 28 May 2024cs.LG cs.CV cs.ROcited by 24

On the map: World models → Humanoid and legged locomotion world models

Abstract (arXiv)

Whole-body control for humanoids is challenging due to the high-dimensional nature of the problem, coupled with the inherent instability of a bipedal morphology. Learning from visual observations further exacerbates this difficulty. In this work, we explore highly data-driven approaches to visual whole-body humanoid control based on reinforcement learning, without any simplifying assumptions, reward design, or skill primitives. Specifically, we propose a hierarchical world model in which a high-level agent generates commands based on visual observations for a low-level agent to execute, both of which are trained with rewards. Our approach produces highly performant control policies in 8 tasks with a simulated 56-DoF humanoid, while synthesizing motions that are broadly preferred by humans.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Humanoid and legged locomotion world models, one of 11 directions in World models. Builds on 39 of its 58 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

39 of 58 references.

References · 39

  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  5. 2023Mastering Diverse Domains through World Modelscited by 1,193
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2020Mastering Atari with Discrete World Modelscited by 990
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  9. 2019Model-Based Reinforcement Learning for Ataricited by 495
  10. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  11. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  12. 2021Mastering Atari Games with Limited Datacited by 235
27 more
  1. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  2. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  3. 2022Online Decision Transformercited by 194
  4. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  5. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  6. 2024Expressive Whole-Body Control for Humanoid Robotscited by 117
  7. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 112
  8. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  9. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  10. 2022Deep Hierarchical Planning from Pixelscited by 96
  11. 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
  12. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  13. 2024Simple Hierarchical Planning with Diffusioncited by 71
  14. 2019Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?cited by 66
  15. 2022MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrationscited by 59
  16. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 55
  17. 2022Skill-based Model-based Reinforcement Learningcited by 50
  18. 2023Finetuning Offline World Models in the Real Worldcited by 39
  19. 2018Hierarchical visuomotor control of humanoidscited by 29
  20. 2023Robust and Versatile Bipedal Jumping Control through Reinforcement Learningcited by 27
  21. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 27
  22. 2022MoCapAct: A Multi-Task Dataset for Simulated Humanoid Controlcited by 24
  23. 2022Joint Embedding Predictive Architectures Focus on Slow Featurescited by 24
  24. 2023Gradient-based Planning with World Modelscited by 18
  25. 2022On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learningcited by 15
  26. 2018SOLAR: Deep Structured Representations for Model-Based Reinforcement Learningcited by 14
  27. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 10.

24 papers.

Sort:

By area:

  1. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  2. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  3. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  4. 2026RISE: Self-Improving Robot Policy with Compositional World Modelcited by 39
  5. 2026Hierarchical Planning with Latent World Modelscited by 34
  6. 2025Whole-Body Conditioned Egocentric Video Predictioncited by 30
  7. 2025UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Controlcited by 24
  8. 2025Embodied AI: From LLMs to World Modelscited by 23
  9. 2025Learning Getting-Up Policies for Real-World Humanoid Robotscited by 21
  10. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 14
  11. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  12. 2026When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Modelscited by 11
12 more
  1. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  2. 2025SkillBlender: Towards Versatile Humanoid Whole-Body Loco-Manipulation via Skill Blendingcited by 9
  3. 2026HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Modelcited by 3
  4. 2026PhyWorld: Physics-Faithful World Model for Video Generationcited by 3
  5. 2024Gazing at Rewards: Eye Movements as a Lens into Human and AI Decision-Making in Hybrid Visual Foragingcited by 2
  6. 2025DyMoDreamer: World Modeling with Dynamic Modulationcited by 2
  7. 2026Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Modelcited by 1
  8. 2026Lifting Embodied World Models for Planning and Controlcited by 1
  9. 2024MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBenchcited by 0
  10. 2026Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learningcited by 0
  11. 2026Neuro-Inspired Inverse Learning for Planning and Controlcited by 0
  12. 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation
  2. 2026WholeBodyWAM: Generalizing Pre-trained World-Action Priors to Humanoid Loco-Manipulation via WBC-Grounded Coordination
  3. 2026Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking
  4. 2025LeVERB: Humanoid Whole-Body Control with Latent Vision-Language Instruction
  5. 2025KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control
  6. 2026HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning
  7. 2025Humanoid World Models: Open World Foundation Models for Humanoid Robotics
  8. 2026Embodiment-Aware Generalist Specialist Distillation for Unified Humanoid Whole-Body Control