# An Adaptable Approach to Learn Realistic Legged Locomotion without Examples

arXiv: https://arxiv.org/abs/2110.14998 · HTML: https://mlsanity.com/paper/2110.14998/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 20 of 20)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation](https://mlsanity.com/paper/1904.13041/) (2019)
- [An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research](https://mlsanity.com/paper/1910.00093/) (2019)
- [Learning Spring Mass Locomotion: Guiding Policies with a Reduced-Order Model](https://mlsanity.com/paper/2010.11234/) (2020)
- [Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1909.12324/) (2019)
- [Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion](https://mlsanity.com/paper/2008.12228/) (2020)
- [Learning and Exploring Motor Skills with Spacetime Bounds](https://mlsanity.com/paper/2103.16807/) (2021)
- [CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2005.03288/) (2020)
- [On Time Optimization of Centroidal Momentum Dynamics](https://mlsanity.com/paper/1709.09265/) (2017)
- [Learning Active Spine Behaviors for Dynamic and Efficient Locomotion in Quadruped Robots](https://mlsanity.com/paper/1905.06077/) (2019)
- [Generalized Maximum Causal Entropy for Inverse Reinforcement Learning](https://mlsanity.com/paper/1911.06928/) (2019)

## Built on by (most cited first, 5 of 5)

- [Leveraging Symmetry in RL-based Legged Locomotion Control](https://mlsanity.com/paper/2403.17320/) (2024)
- [Morphological Symmetries in Robotics](https://mlsanity.com/paper/2402.15552/) (2024)
- [On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis](https://mlsanity.com/paper/2302.10433/) (2023)
- [Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation](https://mlsanity.com/paper/2505.05287/) (2025)
- [Neural Circuit Architectural Priors for Quadruped Locomotion](https://mlsanity.com/paper/2410.07174/) (2024)

## Abstract (arXiv)

Learning controllers that reproduce legged locomotion in nature has been a long-time goal in robotics and computer graphics. While yielding promising results, recent approaches are not yet flexible enough to be applicable to legged systems of different morphologies. This is partly because they often rely on precise motion capture references or elaborate learning environments that ensure the naturality of the emergent locomotion gaits but prevent generalization. This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. Leveraging on the exploration capacities of Reinforcement Learning (RL), we learn a control policy that fills in the information gap between the template model and full-body dynamics required to maintain stable and periodic locomotion. The proposed approach can be applied to robots of different sizes and morphologies and adapted to any RL technique and control architecture. We present experimental results showing that even in a model-free setup and with a simple reactive control architecture, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot. And most importantly, this is achieved without using motion capture, strong constraints in the dynamics or kinematics of the robot, nor prescribing limb coordination. We provide supplemental videos for qualitative analysis of the naturality of the learned gaits.