# WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots

arXiv: https://arxiv.org/abs/2607.24036 · HTML: https://mlsanity.com/paper/2607.24036/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning](https://mlsanity.com/paper/1810.05762/) (2018)
- [Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks](https://mlsanity.com/paper/1906.08880/) (2019)
- [CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion](https://mlsanity.com/paper/2211.00458/) (2022)
- [Robust Quadruped Jumping via Deep Reinforcement Learning](https://mlsanity.com/paper/2011.07089/) (2020)
- [On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer](https://mlsanity.com/paper/2312.03673/) (2023)
- [Learning-based legged locomotion; state of the art and future perspectives](https://mlsanity.com/paper/2406.01152/) (2024)
- [Learning Task Space Actions for Bipedal Locomotion](https://mlsanity.com/paper/2011.04741/) (2020)
- [High-speed control and navigation for quadrupedal robots on complex and discrete terrain](https://mlsanity.com/paper/2506.02835/) (2025)
- [Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Design](https://mlsanity.com/paper/2401.16337/) (2024)
- [ZEST: Zero-shot Embodied Skill Transfer for Athletic Robot Control](https://mlsanity.com/paper/2602.00401/) (2026)
- [Robust Ladder Climbing with a Quadrupedal Robot](https://mlsanity.com/paper/2409.17731/) (2024)
- [Learning Whole-Body Loco-Manipulation for Omni-Directional Task Space Pose Tracking with a Wheeled-Quadrupedal-Manipulator](https://mlsanity.com/paper/2412.03012/) (2024)
- [Learning Motion Skills with Adaptive Assistive Curriculum Force in Humanoid Robots](https://mlsanity.com/paper/2506.23125/) (2025)
- [KLEIYN : A Quadruped Robot with an Active Waist for Both Locomotion and Wall Climbing](https://mlsanity.com/paper/2507.06562/) (2025)
- [EFGCL: Learning Dynamic Motion through Spotting-Inspired External Force Guided Curriculum Learning](https://mlsanity.com/paper/2605.10063/) (2026)

## Abstract (arXiv)

While reinforcement learning for legged robots has achieved high motor performance, it has been constrained by the limited exploration capability of actions confined to the joint space. To address this issue, this study proposes a new method, Wrench-Augmented Reinforcement Learning (WARL), which introduces a wrenche (force and torque) into the action space. The proposed method combines wrench-guided exploration with a success rate-based curriculum mechanism to expand exploration capabilities in the early stages of learning, with the ultimate goal of acquiring behaviors based solely on joint control. Experiments using a quadruped robot demonstrated that WARL can learn robustly across diverse terrains and motor tasks without requiring terrain-specific reward adjustments or complex curriculum designs. Furthermore, an ablation study verified the effectiveness of the Switching Curriculum, which gradually eliminates the wrench. On the other hand, we also show that introducing a wrench can encourage behaviors that do not sufficiently exploit the robot's physical embodiment. These findings suggest that while wrench-based exploration enhancement is effective for improving learning efficiency, designing it in a way that is consistent with the robot's physical structure is a critical future challenge.