# DeepCPG Policies for Robot Locomotion

arXiv: https://arxiv.org/abs/2302.13191 · HTML: https://mlsanity.com/paper/2302.13191/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 88 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [The Limits and Potentials of Deep Learning for Robotics](https://mlsanity.com/paper/1804.06557/) (2018)
- [Neural Dynamic Policies for End-to-End Sensorimotor Learning](https://mlsanity.com/paper/2012.02788/) (2020)
- [CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion](https://mlsanity.com/paper/2211.00458/) (2022)
- [Imitation Learning from MPC for Quadrupedal Multi-Gait Control](https://mlsanity.com/paper/2103.14331/) (2021)
- [Meta Reinforcement Learning for Optimal Design of Legged Robots](https://mlsanity.com/paper/2210.02750/) (2022)
- [Structured Control Nets for Deep Reinforcement Learning](https://mlsanity.com/paper/1802.08311/) (2018)
- [On-line and on-board planning and perception for quadrupedal locomotion](https://mlsanity.com/paper/1904.03693/) (2019)
- [Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot](https://mlsanity.com/paper/2005.11164/) (2020)
- [CPG-ACTOR: Reinforcement Learning for Central Pattern Generators](https://mlsanity.com/paper/2102.12891/) (2021)
- [Evolved embodied phase coordination enables robust quadruped robot locomotion](https://mlsanity.com/paper/1904.03855/) (2019)

## Built on by (most cited first, 4 of 4)

- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Reinforcement Learning-Based Model Matching to Reduce the Sim-Real Gap in COBRA](https://mlsanity.com/paper/2406.13700/) (2024)
- [An Interpretable Neural Control Network with Adaptable Online Learning for Sample Efficient Robot Locomotion Learning](https://mlsanity.com/paper/2501.10698/) (2025)
- [Model-Based Reinforcement Learning Exploits Passive Body Dynamics for High-Performance Biped Robot Locomotion](https://mlsanity.com/paper/2604.14565/) (2026)

## Abstract (arXiv)

Central Pattern Generators (CPGs) form the neural basis of the observed rhythmic behaviors for locomotion in legged animals. The CPG dynamics organized into networks allow the emergence of complex locomotor behaviors. In this work, we take this inspiration for developing walking behaviors in multi-legged robots. We present novel DeepCPG policies that embed CPGs as a layer in a larger neural network and facilitate end-to-end learning of locomotion behaviors in deep reinforcement learning (DRL) setup. We demonstrate the effectiveness of this approach on physics engine-based insectoid robots. We show that, compared to traditional approaches, DeepCPG policies allow sample-efficient end-to-end learning of effective locomotion strategies even in the case of high-dimensional sensor spaces (vision). We scale the DeepCPG policies using a modular robot configuration and multi-agent DRL. Our results suggest that gradual complexification with embedded priors of these policies in a modular fashion could achieve non-trivial sensor and motor integration on a robot platform. These results also indicate the efficacy of bootstrapping more complex intelligent systems from simpler ones based on biological principles. Finally, we present the experimental results for a proof-of-concept insectoid robot system for which DeepCPG learned policies initially using the simulation engine and these were afterwards transferred to real-world robots without any additional fine-tuning.