# Reinforcement Learning For Quadrupedal Locomotion: Current Advancements And Future Perspectives

arXiv: https://arxiv.org/abs/2410.10438 · HTML: https://mlsanity.com/paper/2410.10438/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 14 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Teacher-Student Curriculum Learning](https://mlsanity.com/paper/1707.00183/) (2017)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Dropout Q-Functions for Doubly Efficient Reinforcement Learning](https://mlsanity.com/paper/2110.02034/) (2021)
- [DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10602/) (2023)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)
- [Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds](https://mlsanity.com/paper/1712.02889/) (2017)
- [Learning Multiple Gaits within Latent Space for Quadruped Robots](https://mlsanity.com/paper/2308.03014/) (2023)
- [Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning](https://mlsanity.com/paper/2403.03848/) (2024)
- [MPC-based Controller with Terrain Insight for Dynamic Legged Locomotion](https://mlsanity.com/paper/1909.13842/) (2019)
- [SYNLOCO: Synthesizing Central Pattern Generator and Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2310.06606/) (2023)
- [ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization](https://mlsanity.com/paper/2403.01928/) (2024)
- [Nonlinear Model Predictive Control for Quadrupedal Locomotion Using Second-Order Sensitivity Analysis](https://mlsanity.com/paper/2207.10465/) (2022)

## Built on by (most cited first, 2 of 2)

- [Learning All-Terrain Locomotion for a Planetary Rover with Actively Articulated Suspension](https://mlsanity.com/paper/2606.06790/) (2026)
- [CopyQuad: Quadrupedal Skill Transfer from a Single Side-View Video Without a Depth Model](https://mlsanity.com/paper/2610.16737/) (2026)

## Abstract (arXiv)

In recent years, reinforcement learning (RL) based quadrupedal locomotion control has emerged as an extensively researched field, driven by the potential advantages of autonomous learning and adaptation compared to traditional control methods. This paper provides a comprehensive study of the latest research in applying RL techniques to develop locomotion controllers for quadrupedal robots. We present a detailed overview of the core concepts, methodologies, and key advancements in RL-based locomotion controllers, including learning algorithms, training curricula, reward formulations, and simulation-to-real transfer techniques. The study covers both gait-bound and gait-free approaches, highlighting their respective strengths and limitations. Additionally, we discuss the integration of these controllers with robotic hardware and the role of sensor feedback in enabling adaptive behavior. The paper also outlines future research directions, such as incorporating exteroceptive sensing, combining model-based and model-free techniques, and developing online learning capabilities. Our study aims to provide researchers and practitioners with a comprehensive understanding of the state-of-the-art in RL-based locomotion controllers, enabling them to build upon existing work and explore novel solutions for enhancing the mobility and adaptability of quadrupedal robots in real-world environments.