# Learning Torque Control for Quadrupedal Locomotion

arXiv: https://arxiv.org/abs/2203.05194 · HTML: https://mlsanity.com/paper/2203.05194/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 84 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 30 of 48)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study](https://mlsanity.com/paper/2006.05990/) (2020)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Learning to Walk in the Real World with Minimal Human Effort](https://mlsanity.com/paper/2002.08550/) (2020)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World](https://mlsanity.com/paper/2110.05457/) (2021)
- [Multi-expert learning of adaptive legged locomotion](https://mlsanity.com/paper/2012.05810/) (2020)
- [MNN: A Universal and Efficient Inference Engine](https://mlsanity.com/paper/2002.12418/) (2020)
- [Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion](https://mlsanity.com/paper/2011.02404/) (2020)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)

## Built on by (most cited first, 29 of 29)

- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion](https://mlsanity.com/paper/2211.00458/) (2022)
- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion](https://mlsanity.com/paper/2212.14400/) (2022)
- [FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robots](https://mlsanity.com/paper/2505.06883/) (2025)
- [DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies](https://mlsanity.com/paper/2310.05714/) (2023)
- [HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation](https://mlsanity.com/paper/2406.14655/) (2024)
- [Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion](https://mlsanity.com/paper/2403.20001/) (2024)
- [Guided Reinforcement Learning for Omnidirectional 3D Jumping in Quadruped Robots](https://mlsanity.com/paper/2507.16481/) (2025)
- [End-to-End Reinforcement Learning for Torque Based Variable Height Hopping](https://mlsanity.com/paper/2307.16676/) (2023)
- [Seeing-Eye Quadruped Navigation with Force Responsive Locomotion Control](https://mlsanity.com/paper/2309.04370/) (2023)
- [Disturbance-Aware Adaptive Compensation in Hybrid Force-Position Locomotion Policy for Legged Robots](https://mlsanity.com/paper/2506.00472/) (2025)
- [AURA: Autonomous Upskilling with Retrieval-Augmented Agents](https://mlsanity.com/paper/2506.02507/) (2025)
- [Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision](https://mlsanity.com/paper/2601.16109/) (2026)
- [Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds](https://mlsanity.com/paper/2607.18135/) (2026)
- [Efficient Reinforcement Learning for Jumping Monopods](https://mlsanity.com/paper/2309.07038/) (2023)
- [Tracking Control for a Spherical Pendulum via Curriculum Reinforcement Learning](https://mlsanity.com/paper/2309.14096/) (2023)
- [Variable Stiffness for Robust Locomotion through Reinforcement Learning](https://mlsanity.com/paper/2502.09436/) (2025)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control](https://mlsanity.com/paper/2601.20668/) (2026)
- [Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2606.18594/) (2026)
- [Agile perceptive multi-skill locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2607.13579/) (2026)
- [Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains](https://mlsanity.com/paper/2405.17227/) (2024)
- [BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO](https://mlsanity.com/paper/2406.17490/) (2024)
- [Latent Action Priors for Locomotion with Deep Reinforcement Learning](https://mlsanity.com/paper/2410.03246/) (2024)
- [Cooperative-Competitive Team Play of Real-World Craft Robots](https://mlsanity.com/paper/2602.21119/) (2026)
- [Towards Batch-to-Streaming Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/2603.08588/) (2026)
- [Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain](https://mlsanity.com/paper/2605.09595/) (2026)
- [Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2607.18365/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) has become a promising approach to developing controllers for quadrupedal robots. Conventionally, an RL design for locomotion follows a position-based paradigm, wherein an RL policy outputs target joint positions at a low frequency that are then tracked by a high-frequency proportional-derivative (PD) controller to produce joint torques. In contrast, for the model-based control of quadrupedal locomotion, there has been a paradigm shift from position-based control to torque-based control. In light of the recent advances in model-based control, we explore an alternative to the position-based RL paradigm, by introducing a torque-based RL framework, where an RL policy directly predicts joint torques at a high frequency, thus circumventing the use of a PD controller. The proposed learning torque control framework is validated with extensive experiments, in which a quadruped is capable of traversing various terrain and resisting external disturbances while following user-specified commands. Furthermore, compared to learning position control, learning torque control demonstrates the potential to achieve a higher reward and is more robust to significant external disturbances. To our knowledge, this is the first sim-to-real attempt for end-to-end learning torque control of quadrupedal locomotion.