# Learning Spring Mass Locomotion: Guiding Policies with a Reduced-Order Model

arXiv: https://arxiv.org/abs/2010.11234 · HTML: https://mlsanity.com/paper/2010.11234/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 15 of 15)

- [MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations](https://mlsanity.com/paper/2310.10198/) (2023)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Benchmarking Potential Based Rewards for Learning Humanoid Locomotion](https://mlsanity.com/paper/2307.10142/) (2023)
- [Integrating Model-Based Footstep Planning with Model-Free Reinforcement Learning for Dynamic Legged Locomotion](https://mlsanity.com/paper/2408.02662/) (2024)
- [Learning to Brachiate via Simplified Model Imitation](https://mlsanity.com/paper/2205.03943/) (2022)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [An Adaptable Approach to Learn Realistic Legged Locomotion without Examples](https://mlsanity.com/paper/2110.14998/) (2021)
- [RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control](https://mlsanity.com/paper/2407.17683/) (2024)
- [CLF-RL: Control Lyapunov Function Guided Reinforcement Learning](https://mlsanity.com/paper/2508.09354/) (2025)
- [Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning](https://mlsanity.com/paper/2509.19573/) (2025)
- [Residual MPC: Blending Reinforcement Learning with GPU-Parallelized Model Predictive Control](https://mlsanity.com/paper/2510.12717/) (2025)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Bipedal Locomotion](https://mlsanity.com/paper/2511.06465/) (2025)
- [Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Styles](https://mlsanity.com/paper/2204.04157/) (2022)
- [Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking](https://mlsanity.com/paper/2312.10328/) (2023)
- [Reduced-Order Model-Guided Reinforcement Learning for Demonstration-Free Humanoid Locomotion](https://mlsanity.com/paper/2509.19023/) (2025)

## Abstract (arXiv)

In this paper, we describe an approach to achieve dynamic legged locomotion on physical robots which combines existing methods for control with reinforcement learning. Specifically, our goal is a control hierarchy in which highest-level behaviors are planned through reduced-order models, which describe the fundamental physics of legged locomotion, and lower level controllers utilize a learned policy that can bridge the gap between the idealized, simple model and the complex, full order robot. The high-level planner can use a model of the environment and be task specific, while the low-level learned controller can execute a wide range of motions so that it applies to many different tasks. In this letter we describe this learned dynamic walking controller and show that a range of walking motions from reduced-order models can be used as the command and primary training signal for learned policies. The resulting policies do not attempt to naively track the motion (as a traditional trajectory tracking controller would) but instead balance immediate motion tracking with long term stability. The resulting controller is demonstrated on a human scale, unconstrained, untethered bipedal robot at speeds up to 1.2 m/s. This letter builds the foundation of a generic, dynamic learned walking controller that can be applied to many different tasks.