# Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

arXiv: https://arxiv.org/abs/2111.00262 · HTML: https://mlsanity.com/paper/2111.00262/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 30 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Neural probabilistic motor primitives for humanoid control](https://mlsanity.com/paper/1811.11711/) (2018)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control](https://mlsanity.com/paper/1909.12238/) (2019)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [A Distributional View on Multi-Objective Policy Optimization](https://mlsanity.com/paper/2005.07513/) (2020)
- [DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning](https://mlsanity.com/paper/1909.08399/) (2019)
- [MPC-Net: A First Principles Guided Policy Search](https://mlsanity.com/paper/1909.05197/) (2019)
- [Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2002.09676/) (2020)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control](https://mlsanity.com/paper/2104.02180/) (2021)
- [Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion](https://mlsanity.com/paper/2109.06409/) (2021)
- [Zero-Shot Terrain Generalization for Visual Locomotion Policies](https://mlsanity.com/paper/2011.05513/) (2020)
- [Reliable Trajectories for Dynamic Quadrupeds using Analytical Costs and Learned Initializations](https://mlsanity.com/paper/2002.06719/) (2020)

## Built on by (most cited first, 4 of 4)

- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [Agile perceptive multi-skill locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2607.13579/) (2026)

## Abstract (arXiv)

Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear solvers can produce coordinated constraint satisfying motions, but often take too long to converge for online application. RL methods can learn dynamic reactive controllers but require carefully tuned shaping rewards to produce good gaits and can have trouble discovering precise coordinated movements. Imitation learning circumvents this problem and has been used with motion capture data to extract quadruped gaits for flat terrains. However, it would be costly to acquire motion capture data for a very large variety of terrains with height differences. In this work, we combine the advantages of trajectory optimization and learning methods and show that terrain adaptive controllers can be obtained by training policies to imitate trajectories that have been planned over procedural terrains by a non-linear solver. We show that the learned policies transfer to unseen terrains and can be fine-tuned to dynamically traverse challenging terrains that require precise foot placements and are very hard to solve with standard RL.