# Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2309.16683 · HTML: https://mlsanity.com/paper/2309.16683/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [OPT-Mimic: Imitation of Optimized Trajectories for Dynamic Quadruped Behaviors](https://mlsanity.com/paper/2210.01247/) (2022)
- [An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research](https://mlsanity.com/paper/1910.00093/) (2019)
- [Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions](https://mlsanity.com/paper/2209.07899/) (2022)
- [Planning and Execution of Dynamic Whole-Body Locomotion for a Hydraulic Quadruped on Challenging Terrain](https://mlsanity.com/paper/1904.03695/) (2019)

## Built on by (most cited first, 16 of 16)

- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience](https://mlsanity.com/paper/2409.13678/) (2024)
- [Verification-Guided Shielding for Deep Reinforcement Learning](https://mlsanity.com/paper/2406.06507/) (2024)
- [Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning](https://mlsanity.com/paper/2412.05515/) (2024)
- [Guided Reinforcement Learning for Omnidirectional 3D Jumping in Quadruped Robots](https://mlsanity.com/paper/2507.16481/) (2025)
- [Realizable Continuous-Space Shields for Safe Reinforcement Learning](https://mlsanity.com/paper/2410.02038/) (2024)
- [Reinforcement Learning to improve delta robot throws for sorting scrap metal](https://mlsanity.com/paper/2406.13453/) (2024)
- [Reinforcement Learning from Wild Animal Videos](https://mlsanity.com/paper/2412.04273/) (2024)
- [Efficient Reinforcement Learning for Jumping Monopods](https://mlsanity.com/paper/2309.07038/) (2023)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [Learn to Swim: Data-Driven LSTM Hydrodynamic Model for Quadruped Robot Gait Optimization](https://mlsanity.com/paper/2505.03146/) (2025)
- [McARL:Morphology-Control-Aware Reinforcement Learning for Generalizable Quadrupedal Locomotion](https://mlsanity.com/paper/2505.18418/) (2025)
- [Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot](https://mlsanity.com/paper/2508.02194/) (2025)
- [Locomotion Generation for a Rat Robot based on Environmental Changes via Reinforcement Learning](https://mlsanity.com/paper/2403.11788/) (2024)
- [Coordinating Spinal and Limb Dynamics for Enhanced Sprawling Robot Mobility](https://mlsanity.com/paper/2504.14103/) (2025)
- [ReTracing: An Archaeological Approach Through Body, Machine, and Generative Systems](https://mlsanity.com/paper/2602.11242/) (2026)

## Abstract (arXiv)

Quadruped robots require robust and general locomotion skills to exploit their mobility potential in complex and challenging environments. In this work, we present the first implementation of a robust end-to-end learning-based controller on the Solo12 quadruped. Our method is based on deep reinforcement learning of joint impedance references. The resulting control policies follow a commanded velocity reference while being efficient in its energy consumption, robust and easy to deploy. We detail the learning procedure and method for transfer on the real robot. In our experiments, we show that the Solo12 robot is a suitable open-source platform for research combining learning and control because of the easiness in transferring and deploying learned controllers.