# safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics

arXiv: https://arxiv.org/abs/2109.06325 · HTML: https://mlsanity.com/paper/2109.06325/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [Benchmarking Model-Based Reinforcement Learning](https://mlsanity.com/paper/1907.02057/) (2019)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [AI Safety Gridworlds](https://mlsanity.com/paper/1711.09883/) (2017)
- [Cautious Model Predictive Control using Gaussian Process Regression](https://mlsanity.com/paper/1705.10702/) (2017)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Learning to Fly -- a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control](https://mlsanity.com/paper/2103.02142/) (2021)
- [Learning for Safety-Critical Control with Control Barrier Functions](https://mlsanity.com/paper/1912.10099/) (2019)
- [Robust Reinforcement Learning using Adversarial Populations](https://mlsanity.com/paper/2008.01825/) (2020)
- [Linear model predictive safety certification for learning-based control](https://mlsanity.com/paper/1803.08552/) (2018)
- [Safe Learning of Quadrotor Dynamics Using Barrier Certificates](https://mlsanity.com/paper/1710.05472/) (2017)
- [Optimal and Learning Control for Autonomous Robots](https://mlsanity.com/paper/1708.09342/) (2017)

## Built on by (most cited first, 30 of 50)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2310.12567/) (2023)
- [SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning](https://mlsanity.com/paper/2503.03480/) (2025)
- [Reachability Constrained Reinforcement Learning](https://mlsanity.com/paper/2205.07536/) (2022)
- [Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability](https://mlsanity.com/paper/2209.08025/) (2022)
- [Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopes](https://mlsanity.com/paper/2210.10691/) (2022)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks](https://mlsanity.com/paper/2212.05727/) (2022)
- [Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificate](https://mlsanity.com/paper/2210.07553/) (2022)
- [Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agents](https://mlsanity.com/paper/2410.11671/) (2024)
- [GUARD: A Safe Reinforcement Learning Benchmark](https://mlsanity.com/paper/2305.13681/) (2023)
- [Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learning](https://mlsanity.com/paper/2502.19652/) (2025)
- [A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safety](https://mlsanity.com/paper/2505.17342/) (2025)
- [SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving](https://mlsanity.com/paper/2206.08528/) (2022)
- [Can we hop in general? A discussion of benchmark selection and design using the Hopper environment](https://mlsanity.com/paper/2410.08870/) (2024)
- [HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agents](https://mlsanity.com/paper/2503.08241/) (2025)
- [Multi-Step Model Predictive Safety Filters: Reducing Chattering by Increasing the Prediction Horizon](https://mlsanity.com/paper/2309.11453/) (2023)
- [Feasible Policy Iteration for Safe Reinforcement Learning](https://mlsanity.com/paper/2304.08845/) (2023)
- [Robust Safe Reinforcement Learning under Adversarial Disturbances](https://mlsanity.com/paper/2310.07207/) (2023)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Towards Fast Safe Online Reinforcement Learning via Policy Finetuning](https://mlsanity.com/paper/2412.04426/) (2024)
- [Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning](https://mlsanity.com/paper/2507.00257/) (2025)
- [ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models](https://mlsanity.com/paper/2606.27079/) (2026)
- [Vision-driven UAV River Following: Benchmarking with Safe Reinforcement Learning](https://mlsanity.com/paper/2409.08511/) (2024)
- [SafeSwarm: Decentralized Safe RL for the Swarm of Drones Landing in Dense Crowds](https://mlsanity.com/paper/2501.07566/) (2025)
- [A Primer on SO(3) Action Representations in Deep Reinforcement Learning](https://mlsanity.com/paper/2510.11103/) (2025)
- [An Energy-Based Mechanism for Compositional Behavior](https://mlsanity.com/paper/2512.04745/) (2025)
- [CRAX: Fast Safe Reinforcement Learning Benchmarking](https://mlsanity.com/paper/2606.20376/) (2026)
- [Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances](https://mlsanity.com/paper/2409.19746/) (2024)
- [ARMOR: Robust Reinforcement Learning-based Control for UAVs under Physical Attacks](https://mlsanity.com/paper/2506.22423/) (2025)

## Abstract (arXiv)

In recent years, both reinforcement learning and learning-based control -- as well as the study of their safety, which is crucial for deployment in real-world robots -- have gained significant traction. However, to adequately gauge the progress and applicability of new results, we need the tools to equitably compare the approaches proposed by the controls and reinforcement learning communities. Here, we propose a new open-source benchmark suite, called safe-control-gym, supporting both model-based and data-based control techniques. We provide implementations for three dynamic systems -- the cart-pole, the 1D, and 2D quadrotor -- and two control tasks -- stabilization and trajectory tracking. We propose to extend OpenAI's Gym API -- the de facto standard in reinforcement learning research -- with (i) the ability to specify (and query) symbolic dynamics and (ii) constraints, and (iii) (repeatably) inject simulated disturbances in the control inputs, state measurements, and inertial properties. To demonstrate our proposal and in an attempt to bring research communities closer together, we show how to use safe-control-gym to quantitatively compare the control performance, data efficiency, and safety of multiple approaches from the fields of traditional control, learning-based control, and reinforcement learning.