# skrl: Modular and Flexible Library for Reinforcement Learning

arXiv: https://arxiv.org/abs/2202.03825 · HTML: https://mlsanity.com/paper/2202.03825/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 42 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [ChainerRL: A Deep Reinforcement Learning Library](https://mlsanity.com/paper/1912.03905/) (2019)
- [MushroomRL: Simplifying Reinforcement Learning Research](https://mlsanity.com/paper/2001.01102/) (2020)
- [rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorch](https://mlsanity.com/paper/1909.01500/) (2019)
- [Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking](https://mlsanity.com/paper/2011.07537/) (2020)
- [Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning](https://mlsanity.com/paper/2103.04616/) (2021)
- [ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning](https://mlsanity.com/paper/2112.05923/) (2021)
- [Efficient Reinforcement Learning Development with RLzoo](https://mlsanity.com/paper/2009.08644/) (2020)

## Built on by (most cited first, 30 of 49)

- [RSL-RL: A Learning Library for Robotics Research](https://mlsanity.com/paper/2509.10771/) (2025)
- [Maximum Entropy Reinforcement Learning via Energy-Based Normalizing Flow](https://mlsanity.com/paper/2405.13629/) (2024)
- [How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equations](https://mlsanity.com/paper/2306.06335/) (2023)
- [Tune to Learn: How Controller Gains Shape Robot Policy Learning](https://mlsanity.com/paper/2604.02523/) (2026)
- [Recurrent Reinforcement Learning with Memoroids](https://mlsanity.com/paper/2402.09900/) (2024)
- [AnyBody: A Benchmark Suite for Cross-Embodiment Manipulation](https://mlsanity.com/paper/2505.14986/) (2025)
- [XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Library](https://mlsanity.com/paper/2312.16248/) (2023)
- [Scalable Multi-Objective Robot Reinforcement Learning through Gradient Conflict Resolution](https://mlsanity.com/paper/2509.14816/) (2025)
- [Enhancing Tactile-based Reinforcement Learning for Robotic Control](https://mlsanity.com/paper/2510.21609/) (2025)
- [PolicyFlow: Policy Optimization with Continuous Normalizing Flow in Reinforcement Learning](https://mlsanity.com/paper/2602.01156/) (2026)
- [Your Vision-Language-Action Model Already Has Attention Heads For Path Deviation Detection](https://mlsanity.com/paper/2603.13782/) (2026)
- [RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control](https://mlsanity.com/paper/2306.03530/) (2023)
- [Craftium: Bridging Flexibility and Efficiency for Rich 3D Single- and Multi-Agent Environments](https://mlsanity.com/paper/2407.03969/) (2024)
- [Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.01522/) (2025)
- [Space Robotics Bench: Robot Learning Beyond Earth](https://mlsanity.com/paper/2509.23328/) (2025)
- [RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation](https://mlsanity.com/paper/2505.14526/) (2025)
- [Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning](https://mlsanity.com/paper/2507.00257/) (2025)
- [CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks](https://mlsanity.com/paper/2509.14380/) (2025)
- [Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos](https://mlsanity.com/paper/2605.23762/) (2026)
- [Find the Fruit: Zero-Shot Sim2Real RL for Occlusion-Aware Plant Manipulation](https://mlsanity.com/paper/2505.16547/) (2025)
- [Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach](https://mlsanity.com/paper/2508.16807/) (2025)
- [Solving Robotics Tasks with Prior Demonstration via Exploration-Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2509.04069/) (2025)
- [roto 2.0: The Robot Tactile Olympiad](https://mlsanity.com/paper/2605.21429/) (2026)
- [Let it Cook: Learning to Wait in Sequential Decision Making](https://mlsanity.com/paper/2608.11511/) (2026)
- [Using Petri Nets as an Integrated Constraint Mechanism for Reinforcement Learning Tasks](https://mlsanity.com/paper/2407.04481/) (2024)
- [Disentangling Recognition and Decision Regrets in Image-Based Reinforcement Learning](https://mlsanity.com/paper/2409.13108/) (2024)
- [A Graph-Based Reinforcement Learning Approach with Frontier Potential Based Reward for Safe Cluttered Environment Exploration](https://mlsanity.com/paper/2504.11907/) (2025)
- [LineFlow: A Framework to Learn Active Control of Production Lines](https://mlsanity.com/paper/2505.06744/) (2025)
- [CiRL: Open-Source Environments for Reinforcement Learning in Circular Economy and Net Zero](https://mlsanity.com/paper/2505.21536/) (2025)
- [ObjectRL: An Object-Oriented Reinforcement Learning Codebase](https://mlsanity.com/paper/2507.03487/) (2025)

## Abstract (arXiv)

skrl is an open-source modular library for reinforcement learning written in Python and designed with a focus on readability, simplicity, and transparency of algorithm implementations. In addition to supporting environments that use the traditional interfaces from OpenAI Gym and DeepMind, it provides the facility to load, configure, and operate NVIDIA Isaac Gym and NVIDIA Omniverse Isaac Gym environments. Furthermore, it enables the simultaneous training of several agents with customizable scopes (subsets of environments among all available ones), which may or may not share resources, in the same run. The library's documentation can be found at https://skrl.readthedocs.io and its source code is available on GitHub at https://github.com/Toni-SM/skrl.