# Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller

arXiv: https://arxiv.org/abs/2308.04462 · HTML: https://mlsanity.com/paper/2308.04462/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 93 references that are in the map.
- First posted on arXiv 8 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [EPOpt: Learning Robust Neural Network Policies Using Model Ensembles](https://mlsanity.com/paper/1610.01283/) (2016)
- [Robust Reinforcement Learning using Adversarial Populations](https://mlsanity.com/paper/2008.01825/) (2020)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [Feedback Control For Cassie With Deep Reinforcement Learning](https://mlsanity.com/paper/1803.05580/) (2018)
- [Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization](https://mlsanity.com/paper/2107.06629/) (2021)
- [Policy Transfer via Kinematic Domain Randomization and Adaptation](https://mlsanity.com/paper/2011.01891/) (2020)
- [Artificial Intelligence for Prosthetics - challenge solutions](https://mlsanity.com/paper/1902.02441/) (2019)
- [Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped](https://mlsanity.com/paper/1809.10811/) (2018)
- [Learning a Control Policy for Fall Prevention on an Assistive Walking Device](https://mlsanity.com/paper/1909.10488/) (2019)
- [Emergence of Human-comparable Balancing Behaviors by Deep Reinforcement Learning](https://mlsanity.com/paper/1809.02074/) (2018)
- [Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance](https://mlsanity.com/paper/2105.03489/) (2021)
- [Design and Evaluation of Torque Compensation Controllers for a Lower Extremity Exoskeleton](https://mlsanity.com/paper/1907.02200/) (2019)

## Built on by (most cited first, 1 of 1)

- [Bipedal Balance Control with Whole-body Musculoskeletal Standing and Falling Simulations](https://mlsanity.com/paper/2506.09383/) (2025)

## Abstract (arXiv)

Balance assessment during physical rehabilitation often relies on rubric-oriented battery tests to score a patient's physical capabilities, leading to subjectivity. While some objective balance assessments exist, they are often limited to tracking the center of pressure (COP), which does not fully capture the whole-body postural stability. This study explores the use of the center of mass (COM) state space and presents a promising avenue for monitoring the balance capabilities in humans. We employ a musculoskeletal model integrated with a balance controller, trained through reinforcement learning (RL), to investigate balancing capabilities. The RL framework consists of two interconnected neural networks governing balance recovery and muscle coordination respectively, trained using Proximal Policy Optimization (PPO) with reference state initialization, early termination, and multiple training strategies. By exploring recovery from random initial COM states (position and velocity) space for a trained controller, we obtain the final BR enclosing successful balance recovery trajectories. Comparing the BRs with analytical postural stability limits from a linear inverted pendulum model, we observe a similar trend in successful COM states but more limited ranges in the recoverable areas. We further investigate the effect of muscle weakness and neural excitation delay on the BRs, revealing reduced balancing capability in different regions. Overall, our approach of learning muscular balance controllers presents a promising new method for establishing balance recovery limits and objectively assessing balance capability in bipedal systems, particularly in humans.