# Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

arXiv: https://arxiv.org/abs/2603.08619 · HTML: https://mlsanity.com/paper/2603.08619/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [Humanoid Parkour Learning](https://mlsanity.com/paper/2406.10759/) (2024)
- [HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots](https://mlsanity.com/paper/2410.21229/) (2024)
- [WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts](https://mlsanity.com/paper/2406.06005/) (2024)
- [Learning Humanoid Standing-up Control across Diverse Postures](https://mlsanity.com/paper/2502.08378/) (2025)
- [Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control](https://mlsanity.com/paper/1909.04947/) (2019)
- [Symmetry Considerations for Learning Task Symmetric Robot Policies](https://mlsanity.com/paper/2403.04359/) (2024)
- [Learning Getting-Up Policies for Real-World Humanoid Robots](https://mlsanity.com/paper/2502.12152/) (2025)
- [Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking](https://mlsanity.com/paper/2404.19173/) (2024)
- [Fast Contact-Implicit Model-Predictive Control](https://mlsanity.com/paper/2107.05616/) (2021)
- [FRASA: An End-to-End Reinforcement Learning Agent for Fall Recovery and Stand Up of Humanoid Robots](https://mlsanity.com/paper/2410.08655/) (2024)
- [Momentum Control with Hierarchical Inverse Dynamics on a Torque-Controlled Humanoid](https://mlsanity.com/paper/1410.7284/) (2014)
- [HiFAR: Multi-Stage Curriculum Learning for High-Dynamics Humanoid Fall Recovery](https://mlsanity.com/paper/2502.20061/) (2025)
- [Angular Momentum about the Contact Point for Control of Bipedal Locomotion: Validation in a LIP-based Controller](https://mlsanity.com/paper/2008.10763/) (2020)
- [Unified Humanoid Fall-Safety Policy from a Few Demonstrations](https://mlsanity.com/paper/2511.07407/) (2025)
- [A Model Predictive Capture Point Control Framework for Robust Humanoid Balancing via Ankle, Hip, and Stepping Strategies](https://mlsanity.com/paper/2307.13243/) (2023)
- [Learning to Recover: Dynamic Reward Shaping with Wheel-Leg Coordination for Fallen Robots](https://mlsanity.com/paper/2506.05516/) (2025)

## Built on by (most cited first, 2 of 2)

- [HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers](https://mlsanity.com/paper/2606.06493/) (2026)
- [Natural Sit-to-Stand Motion Synthesis For Humanoids via Guided Assistance Curricula and Staged Rewards](https://mlsanity.com/paper/2608.20823/) (2026)

## Abstract (arXiv)

Humanoid robots remain vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments. While reinforcement learning has demonstrated stand-up behaviors, existing approaches treat recovery as a pure task-reward problem without an explicit representation of the balance state. We present a unified RL policy that addresses this limitation by embedding classical balance metrics: capture point, center-of-mass state, and centroidal momentum, as privileged critic inputs and shaping rewards directly around these quantities during training, while the actor relies solely on proprioception for zero-shot hardware transfer. Without reference trajectories or scripted contacts, a single policy spans the full recovery spectrum: ankle and hip strategies for small disturbances, corrective stepping under large pushes, and compliant falling with multi-contact stand-up using the hands, elbows, and knees. Trained on the Unitree H1-2 in Isaac Lab, the policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations. An ablation study shows that removing the balance-informed structure causes stand-up learning to fail entirely, confirming that these metrics provide a meaningful learning signal rather than incidental structure. Sim-to-sim transfer to MuJoCo and preliminary hardware experiments further demonstrate cross-environment generalization. These results show that embedding interpretable balance structure into the learning framework substantially reduces time spent in failure states and broadens the envelope of autonomous recovery.