# DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies

arXiv: https://arxiv.org/abs/2310.05714 · HTML: https://mlsanity.com/paper/2310.05714/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Placed in

- Offline RL and imitation learning from data / [Real-world robot offline RL](https://mlsanity.com/offlinerl/real-world-robot-offline-rl/)

## Builds on (most cited first, 16 of 16)

- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [OPT-Mimic: Imitation of Optimized Trajectories for Dynamic Quadruped Behaviors](https://mlsanity.com/paper/2210.01247/) (2022)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [Learning Modular Robot Control Policies](https://mlsanity.com/paper/2105.10049/) (2021)
- [Torque-based Deep Reinforcement Learning for Task-and-Robot Agnostic Learning on Bipedal Robots Using Sim-to-Real Transfer](https://mlsanity.com/paper/2304.09434/) (2023)
- [Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped](https://mlsanity.com/paper/1709.06047/) (2017)
- [Representation-Free Model Predictive Control for Dynamic Motions in Quadrupeds](https://mlsanity.com/paper/2012.10002/) (2020)
- [SLoMo: A General System for Legged Robot Motion Imitation from Casual Videos](https://mlsanity.com/paper/2304.14389/) (2023)

## Built on by (most cited first, 7 of 7)

- [FACET: Force-Adaptive Control via Impedance Reference Tracking for Legged Robots](https://mlsanity.com/paper/2505.06883/) (2025)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)
- [GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control](https://mlsanity.com/paper/2601.20668/) (2026)
- [Latent Action Priors for Locomotion with Deep Reinforcement Learning](https://mlsanity.com/paper/2410.03246/) (2024)
- [APEX: Adaptive Policy Execution for Precise Manipulation](https://mlsanity.com/paper/2606.16504/) (2026)
- [OmniMimic: Dynamics-completed Motion Augmentation for Multi-style Omnidirectional Quadruped Locomotion](https://mlsanity.com/paper/2609.20566/) (2026)
- [Action Shaping: Policies Absorb What They Can Express](https://mlsanity.com/paper/2609.32752/) (2026)

## Abstract (arXiv)

Optimal Control for legged robots has gone through a paradigm shift from position-based to torque-based control, owing to the latter's compliant and robust nature. In parallel to this shift, the community has also turned to Deep Reinforcement Learning (DRL) as a promising approach to directly learn locomotion policies for complex real-life tasks. However, most end-to-end DRL approaches still operate in position space, mainly because learning in torque space is often sample-inefficient and does not consistently converge to natural gaits. To address these challenges, we propose a two-stage framework. In the first stage, we generate our own imitation data by training a position-based policy, eliminating the need for expert knowledge to design optimal controllers. The second stage incorporates decaying action priors, a novel method to enhance the exploration of torque-based policies aided by imitation rewards. We show that our approach consistently outperforms imitation learning alone and is robust to scaling these rewards from 0.1x to 10x. We further validate the benefits of torque control by comparing the robustness of a position-based policy to a position-assisted torque-based policy on a quadruped (Unitree Go1) without any domain randomization in the form of external disturbances during training.