# RSL-RL: A Learning Library for Robotics Research

arXiv: https://arxiv.org/abs/2509.10771 · HTML: https://mlsanity.com/paper/2509.10771/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots](https://mlsanity.com/paper/2410.21229/) (2024)
- [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025)
- [Penalized Proximal Policy Optimization for Safe Reinforcement Learning](https://mlsanity.com/paper/2205.11814/) (2022)
- [WoCoCo: Learning Whole-Body Humanoid Control with Sequential Contacts](https://mlsanity.com/paper/2406.06005/) (2024)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [Attention-Based Map Encoding for Learning Generalized Legged Locomotion](https://mlsanity.com/paper/2506.09588/) (2025)
- [Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning](https://mlsanity.com/paper/2203.14912/) (2022)
- [Symmetry Considerations for Learning Task Symmetric Robot Policies](https://mlsanity.com/paper/2403.04359/) (2024)
- [Pedipulate: Enabling Manipulation Skills using a Quadruped Robot's Leg](https://mlsanity.com/paper/2402.10837/) (2024)
- [Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuning](https://mlsanity.com/paper/2505.11164/) (2025)
- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [Learning Agile Locomotion on Risky Terrains](https://mlsanity.com/paper/2311.10484/) (2023)
- [Guided Reinforcement Learning for Robust Multi-Contact Loco-Manipulation](https://mlsanity.com/paper/2410.13817/) (2024)
- [MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.20326/) (2024)
- [Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning](https://mlsanity.com/paper/2502.01546/) (2025)

## Built on by (most cited first, 30 of 49)

- [Learning Sim-to-Real Humanoid Locomotion in 15 Minutes](https://mlsanity.com/paper/2512.01996/) (2025)
- [Flow Policy Gradients for Robot Control](https://mlsanity.com/paper/2602.02481/) (2026)
- [Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer](https://mlsanity.com/paper/2512.01061/) (2025)
- [SoftMimic: Learning Compliant Whole-body Control from Examples](https://mlsanity.com/paper/2510.17792/) (2025)
- [FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructions](https://mlsanity.com/paper/2601.12799/) (2026)
- [From Generated Human Videos to Physically Plausible Robot Trajectories](https://mlsanity.com/paper/2512.05094/) (2025)
- [Scalable Multi-Objective Robot Reinforcement Learning through Gradient Conflict Resolution](https://mlsanity.com/paper/2509.14816/) (2025)
- [PolicyFlow: Policy Optimization with Continuous Normalizing Flow in Reinforcement Learning](https://mlsanity.com/paper/2602.01156/) (2026)
- [FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control](https://mlsanity.com/paper/2604.04539/) (2026)
- [Scaling Rough Terrain Locomotion with Automatic Curriculum Reinforcement Learning](https://mlsanity.com/paper/2601.17428/) (2026)
- [SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flows](https://mlsanity.com/paper/2602.09580/) (2026)
- [PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball](https://mlsanity.com/paper/2607.28623/) (2026)
- [PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations](https://mlsanity.com/paper/2512.13093/) (2025)
- [ViserDex: Visual Sim-to-Real for Robust Dexterous In-hand Reorientation](https://mlsanity.com/paper/2604.11138/) (2026)
- [HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers](https://mlsanity.com/paper/2606.06493/) (2026)
- [Trust-Region Diffusion Policies for Massively Parallel On-Policy RL](https://mlsanity.com/paper/2606.15260/) (2026)
- [WaveLander: A Generalizable Hierarchical Control Framework for UAV Landing on Wave-Disturbed Platforms via Reinforcement Learning](https://mlsanity.com/paper/2607.01281/) (2026)
- [End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2509.18608/) (2025)
- [Towards Bridging the Gap between Large-Scale Pretraining and Efficient Finetuning for Humanoid Control](https://mlsanity.com/paper/2601.21363/) (2026)
- [Learning Policy Representations for Steerable Behavior Synthesis](https://mlsanity.com/paper/2601.22350/) (2026)
- [NFPO: Stabilized Policy Optimization of Normalizing Flow for Robotic Policy Learning](https://mlsanity.com/paper/2603.11470/) (2026)
- [Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient](https://mlsanity.com/paper/2605.26478/) (2026)
- [When Is a Learned Command Adapter Worth It? Closed-Loop Identification and Counterfactual Auditing of Frozen Locomotion Policies](https://mlsanity.com/paper/2607.21867/) (2026)
- [Pretraining in Actor-Critic Reinforcement Learning for Locomotion](https://mlsanity.com/paper/2510.12363/) (2025)
- [Learning End-to-End Control for Omnidirectional Aerial Motion on Overactuated Tilt-rotor Quadrotors](https://mlsanity.com/paper/2602.21583/) (2026)
- [Path-conditioned Reinforcement Learning-based Local Planning for Long-Range Navigation](https://mlsanity.com/paper/2603.13888/) (2026)
- [PriPG-RL: Privileged Planner-Guided Reinforcement Learning for Partially Observable Systems with Anytime-Feasible MPC](https://mlsanity.com/paper/2604.08036/) (2026)
- [Bounded Ratio Reinforcement Learning](https://mlsanity.com/paper/2604.18578/) (2026)
- [Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies](https://mlsanity.com/paper/2605.05373/) (2026)
- [Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy](https://mlsanity.com/paper/2605.15517/) (2026)

## Abstract (arXiv)

RSL-RL is an open-source Reinforcement Learning library tailored to the specific needs of the robotics community. Unlike broad general-purpose frameworks, its design philosophy prioritizes a compact and easily modifiable codebase, allowing researchers to adapt and extend algorithms with minimal overhead. The library focuses on algorithms most widely adopted in robotics, together with auxiliary techniques that address robotics-specific challenges. Optimized for GPU-only training, RSL-RL achieves high-throughput performance in large-scale simulation environments. Its effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments, demonstrating its utility as a lightweight, extensible, and practical framework to develop learning-based robotic controllers. The library is open-sourced at: https://github.com/leggedrobotics/rsl_rl.