# From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots

arXiv: https://arxiv.org/abs/2506.12779 · HTML: https://mlsanity.com/paper/2506.12779/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 15 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation](https://mlsanity.com/paper/2403.04436/) (2024)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [Deep Learning for Detecting Robotic Grasps](https://mlsanity.com/paper/1301.3592/) (2013)
- [HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots](https://mlsanity.com/paper/2410.21229/) (2024)
- [ExBody2: Advanced Expressive Humanoid Whole-Body Control](https://mlsanity.com/paper/2412.13196/) (2024)
- [Humanoid Locomotion as Next Token Prediction](https://mlsanity.com/paper/2402.19469/) (2024)
- [AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control](https://mlsanity.com/paper/2505.03738/) (2025)
- [Learning Fast Adaptation with Meta Strategy Optimization](https://mlsanity.com/paper/1909.12995/) (2019)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Sim-to-Real Learning for Humanoid Box Loco-Manipulation](https://mlsanity.com/paper/2310.03191/) (2023)
- [The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors](https://mlsanity.com/paper/2104.09025/) (2021)
- [Transformer-based deep imitation learning for dual-arm robot manipulation](https://mlsanity.com/paper/2108.00385/) (2021)
- [Robotic Grasping using Deep Reinforcement Learning](https://mlsanity.com/paper/2007.04499/) (2020)
- [Whole-Body Geometric Retargeting for Humanoid Robots](https://mlsanity.com/paper/1909.10080/) (2019)
- [Robocentric Visual-Inertial Odometry](https://mlsanity.com/paper/1805.04031/) (2018)
- [Agile Continuous Jumping in Discontinuous Terrains](https://mlsanity.com/paper/2409.10923/) (2024)
- [JAEGER: Dual-Level Humanoid Whole-Body Controller](https://mlsanity.com/paper/2505.06584/) (2025)

## Built on by (most cited first, 17 of 17)

- [KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control](https://mlsanity.com/paper/2509.16638/) (2025)
- [DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation](https://mlsanity.com/paper/2510.11258/) (2025)
- [HoloMotion-1 Technical Report](https://mlsanity.com/paper/2605.15336/) (2026)
- [SENTINEL: A Fully End-to-End Language-Action Model for Humanoid Whole Body Control](https://mlsanity.com/paper/2511.19236/) (2025)
- [RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion](https://mlsanity.com/paper/2512.23649/) (2025)
- [Humanoid-GPT: Scaling Data and Structure for Zero-Shot Motion Tracking](https://mlsanity.com/paper/2606.03985/) (2026)
- [Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Control](https://mlsanity.com/paper/2507.05674/) (2025)
- [RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking](https://mlsanity.com/paper/2509.20717/) (2025)
- [PhyGile: Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking](https://mlsanity.com/paper/2603.19305/) (2026)
- [HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers](https://mlsanity.com/paper/2606.06493/) (2026)
- [Switch: Learning Agile Skills Switching for Humanoid Robots](https://mlsanity.com/paper/2604.14834/) (2026)
- [AnyBody: Free-Form Whole-Body Humanoid Control from Arbitrary Keypoint Guidance](https://mlsanity.com/paper/2606.29209/) (2026)
- [MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2603.08572/) (2026)
- [ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments](https://mlsanity.com/paper/2607.02034/) (2026)
- [Behavior Foundations for Quadruped Robots: ABot-C0 Technical Report](https://mlsanity.com/paper/2607.07370/) (2026)
- [EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal](https://mlsanity.com/paper/2607.13472/) (2026)
- [GigaBrain-WBC-0.5: A Behavior World Model for Robust Humanoid Whole-Body Tracking with Environment Interaction](https://mlsanity.com/paper/2608.18234/) (2026)

## Abstract (arXiv)

Achieving general agile whole-body control on humanoid robots remains a major challenge due to diverse motion demands and data conflicts. While existing frameworks excel in training single motion-specific policies, they struggle to generalize across highly varied behaviors due to conflicting control requirements and mismatched data distributions. In this work, we propose BumbleBee (BB), an expert-generalist learning framework that combines motion clustering and sim-to-real adaptation to overcome these challenges. BB first leverages an autoencoder-based clustering method to group behaviorally similar motions using motion features and motion descriptions. Expert policies are then trained within each cluster and refined with real-world data through iterative delta action modeling to bridge the sim-to-real gap. Finally, these experts are distilled into a unified generalist controller that preserves agility and robustness across all motion types. Experiments on two simulations and a real humanoid robot demonstrate that BB achieves state-of-the-art general whole-body control, setting a new benchmark for agile, robust, and generalizable humanoid performance in the real world. The project webpage is available at https://beingbeyond.github.io/BumbleBee/.