# Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion

arXiv: https://arxiv.org/abs/2002.09676 · HTML: https://mlsanity.com/paper/2002.09676/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2020.

## Built on by (most cited first, 25 of 25)

- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model](https://mlsanity.com/paper/2104.09771/) (2021)
- [Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion](https://mlsanity.com/paper/2209.14887/) (2022)
- [Efficient Off-Policy Safe Reinforcement Learning Using Trust Region Conditional Value at Risk](https://mlsanity.com/paper/2312.00342/) (2023)
- [Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion](https://mlsanity.com/paper/2405.19452/) (2024)
- [TRC: Trust Region Conditional Value at Risk for Safe Reinforcement Learning](https://mlsanity.com/paper/2312.00344/) (2023)
- [I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning](https://mlsanity.com/paper/2405.08726/) (2024)
- [Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner](https://mlsanity.com/paper/2111.00262/) (2021)
- [Bridging the gap between Learning-to-plan, Motion Primitives and Safe Reinforcement Learning](https://mlsanity.com/paper/2408.14063/) (2024)
- [Offline Adaptation of Quadruped Locomotion using Diffusion Models](https://mlsanity.com/paper/2411.08832/) (2024)
- [Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning](https://mlsanity.com/paper/2107.02955/) (2021)
- [Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion](https://mlsanity.com/paper/2112.04809/) (2021)
- [A Transferable Legged Mobile Manipulation Framework Based on Disturbance Predictive Control](https://mlsanity.com/paper/2203.03391/) (2022)
- [VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation](https://mlsanity.com/paper/2205.01179/) (2022)
- [Simplex-enabled Safe Continual Learning Machine](https://mlsanity.com/paper/2409.05898/) (2024)
- [Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds](https://mlsanity.com/paper/2607.18135/) (2026)
- [OGMP: Oracle Guided Multi-mode Policies for Agile and Versatile Robot Control](https://mlsanity.com/paper/2403.04205/) (2024)
- [Learning Rapid Turning, Aerial Reorientation, and Balancing using Manipulator as a Tail](https://mlsanity.com/paper/2407.10420/) (2024)
- [Hierarchical Reinforcement Learning and Value Optimization for Challenging Quadruped Locomotion](https://mlsanity.com/paper/2506.20036/) (2025)
- [Adversarial Constrained Policy Optimization: Improving Constrained Reinforcement Learning by Adapting Budgets](https://mlsanity.com/paper/2410.20786/) (2024)
- [PALo: Learning Posture-Aware Locomotion for Quadruped Robots](https://mlsanity.com/paper/2503.04462/) (2025)
- [Adaptive MPC-based quadrupedal robot control under periodic disturbances](https://mlsanity.com/paper/2505.12361/) (2025)
- [LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors](https://mlsanity.com/paper/2606.15896/) (2026)
- [Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2607.18365/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) uses model-free techniques to optimize task-specific control policies. Despite having emerged as a promising approach for complex problems, RL is still hard to use reliably for real-world applications. Apart from challenges such as precise reward function tuning, inaccurate sensing and actuation, and non-deterministic response, existing RL methods do not guarantee behavior within required safety constraints that are crucial for real robot scenarios. In this regard, we introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained proximal policy optimization (CPPO) for tracking base velocity commands while following the defined constraints. We also introduce schemes which encourage state recovery into constrained regions in case of constraint violations. We present experimental results of our training method and test it on the real ANYmal quadruped robot. We compare our approach against the unconstrained RL method and show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.