# CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

arXiv: https://arxiv.org/abs/2211.00458 · HTML: https://mlsanity.com/paper/2211.00458/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 1 Nov 2022.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Policies Modulating Trajectory Generators](https://mlsanity.com/paper/1910.02812/) (2019)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)
- [Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller](https://mlsanity.com/paper/2201.00206/) (2022)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [Robust Quadruped Jumping via Deep Reinforcement Learning](https://mlsanity.com/paper/2011.07089/) (2020)
- [Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning](https://mlsanity.com/paper/2103.06484/) (2021)
- [Reinforcement Learning with Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion](https://mlsanity.com/paper/2109.06409/) (2021)
- [Training in Task Space to Speed Up and Guide Reinforcement Learning](https://mlsanity.com/paper/1903.02219/) (2019)
- [Adaptive Force-based Control for Legged Robots](https://mlsanity.com/paper/2011.06236/) (2020)
- [A CPG-Based Agile and Versatile Locomotion Framework Using Proximal Symmetry Loss](https://mlsanity.com/paper/2103.00928/) (2021)

## Built on by (most cited first, 30 of 34)

- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [Symmetry Considerations for Learning Task Symmetric Robot Policies](https://mlsanity.com/paper/2403.04359/) (2024)
- [Leveraging Symmetry in RL-based Legged Locomotion Control](https://mlsanity.com/paper/2403.17320/) (2024)
- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots](https://mlsanity.com/paper/2310.10486/) (2023)
- [Learning to Walk from Three Minutes of Real-World Data with Semi-structured Dynamics Models](https://mlsanity.com/paper/2410.09163/) (2024)
- [Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion](https://mlsanity.com/paper/2212.14400/) (2022)
- [Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation](https://mlsanity.com/paper/2410.03076/) (2024)
- [A Learning Framework for Diverse Legged Robot Locomotion Using Barrier-Based Style Rewards](https://mlsanity.com/paper/2409.15780/) (2024)
- [Identifying Important Sensory Feedback for Learning Locomotion Skills](https://mlsanity.com/paper/2306.17101/) (2023)
- [Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drive](https://mlsanity.com/paper/2302.13378/) (2023)
- [DeepCPG Policies for Robot Locomotion](https://mlsanity.com/paper/2302.13191/) (2023)
- [Disturbance-Aware Adaptive Compensation in Hybrid Force-Position Locomotion Policy for Legged Robots](https://mlsanity.com/paper/2506.00472/) (2025)
- [DeepTransition: Viability Leads to the Emergence of Gait Transitions in Learning Anticipatory Quadrupedal Locomotion Skills](https://mlsanity.com/paper/2306.07419/) (2023)
- [Neural Circuit Architectural Priors for Quadruped Locomotion](https://mlsanity.com/paper/2410.07174/) (2024)
- [Coordinated Humanoid Robot Locomotion with Symmetry Equivariant Reinforcement Learning Policy](https://mlsanity.com/paper/2508.01247/) (2025)
- [Learning Emergent Gaits with Decentralized Phase Oscillators: on the role of Observations, Rewards, and Feedback](https://mlsanity.com/paper/2402.08662/) (2024)
- [Behavior evolution-inspired approach to walking gait reinforcement training for quadruped robots](https://mlsanity.com/paper/2409.16862/) (2024)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control](https://mlsanity.com/paper/2601.20668/) (2026)
- [Agile perceptive multi-skill locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2607.13579/) (2026)
- [A Central Motor System Inspired Pre-training Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2311.07822/) (2023)
- [Hierarchical learning control for autonomous robots inspired by central nervous system](https://mlsanity.com/paper/2408.03525/) (2024)
- [Latent Action Priors for Locomotion with Deep Reinforcement Learning](https://mlsanity.com/paper/2410.03246/) (2024)
- [SpikingSoft: A Spiking Neuron Controller for Bio-inspired Locomotion with Soft Snake Robots](https://mlsanity.com/paper/2501.19072/) (2025)
- [PALo: Learning Posture-Aware Locomotion for Quadruped Robots](https://mlsanity.com/paper/2503.04462/) (2025)
- [RANDPOL: Parameter-Efficient End-to-End Quadruped Locomotion via Randomized Policy Learning](https://mlsanity.com/paper/2505.19054/) (2025)
- [PGTT: Phase-Guided Terrain Traversal for Perceptive Legged Locomotion](https://mlsanity.com/paper/2510.18348/) (2025)
- [Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrization](https://mlsanity.com/paper/2604.20365/) (2026)
- [Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain](https://mlsanity.com/paper/2605.09595/) (2026)

## Abstract (arXiv)

In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training, most notably to a dynamically added 13.75 kg load representing 115% of the nominal quadruped mass. We test several different observation spaces based on proprioceptive sensing and show that our framework is deployable with no domain randomization and very little feedback, where along with the oscillator states, it is possible to provide only contact booleans in the observation space. Video results can be found at https://youtu.be/xqXHLzLsEV4.