# SAMBA: Safe Model-Based & Active Reinforcement Learning

arXiv: https://arxiv.org/abs/2006.09436 · HTML: https://mlsanity.com/paper/2006.09436/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 30)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation](https://mlsanity.com/paper/2202.06558/) (2022)
- [Safe Reinforcement Learning Using Robust Control Barrier Functions](https://mlsanity.com/paper/2110.05415/) (2021)
- [Revisiting Design Choices in Offline Model-Based Reinforcement Learning](https://mlsanity.com/paper/2110.04135/) (2021)
- [Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithm](https://mlsanity.com/paper/2210.07573/) (2022)
- [Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction](https://mlsanity.com/paper/2209.13308/) (2022)
- [Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications](https://mlsanity.com/paper/2404.09080/) (2024)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Provable Safe Reinforcement Learning with Binary Feedback](https://mlsanity.com/paper/2210.14492/) (2022)
- [Effects of Safety State Augmentation on Safe Exploration](https://mlsanity.com/paper/2206.02675/) (2022)
- [Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2303.03955/) (2023)
- [Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention](https://mlsanity.com/paper/2111.05819/) (2021)
- [Learning Collision-free and Torque-limited Robot Trajectories based on Alternative Safe Behaviors](https://mlsanity.com/paper/2103.03793/) (2021)
- [DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention](https://mlsanity.com/paper/2110.14468/) (2021)
- [Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles](https://mlsanity.com/paper/2112.11217/) (2021)
- [ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization](https://mlsanity.com/paper/2403.01928/) (2024)
- [Risk Sensitive Model-Based Reinforcement Learning using Uncertainty Guided Planning](https://mlsanity.com/paper/2111.04972/) (2021)
- [Handling Long-Term Safety and Uncertainty in Safe Reinforcement Learning](https://mlsanity.com/paper/2409.12045/) (2024)
- [Demystifying Reinforcement Learning in Time-Varying Systems](https://mlsanity.com/paper/2201.05560/) (2022)
- [Improving Controller Generalization with Dimensionless Markov Decision Processes](https://mlsanity.com/paper/2504.10006/) (2025)
- [Learning Robust Controllers Via Probabilistic Model-Based Policy Search](https://mlsanity.com/paper/2110.13576/) (2021)
- [Guided Safe Shooting: model based reinforcement learning with safety constraints](https://mlsanity.com/paper/2206.09743/) (2022)
- [Recent Trends in Artificial Intelligence Technology: A Scoping Review](https://mlsanity.com/paper/2305.04532/) (2023)
- [Modeling Risk in Reinforcement Learning: A Literature Mapping](https://mlsanity.com/paper/2312.05231/) (2023)
- [Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes](https://mlsanity.com/paper/2405.10581/) (2024)
- [Scaling CrossQ with Weight Normalization](https://mlsanity.com/paper/2506.03758/) (2025)
- [Adaptive Aggregation for Safety-Critical Control](https://mlsanity.com/paper/2302.03586/) (2023)
- [Diverse Priors for Deep Reinforcement Learning](https://mlsanity.com/paper/2310.14864/) (2023)
- [GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model](https://mlsanity.com/paper/2406.03912/) (2024)
- [Diminishing Return of Value Expansion Methods](https://mlsanity.com/paper/2412.20537/) (2024)

## Abstract (arXiv)

In this paper, we propose SAMBA, a novel framework for safe reinforcement learning that combines aspects from probabilistic modelling, information theory, and statistics. Our method builds upon PILCO to enable active exploration using novel(semi-)metrics for out-of-sample Gaussian process evaluation optimised through a multi-objective problem that supports conditional-value-at-risk constraints. We evaluate our algorithm on a variety of safe dynamical system benchmarks involving both low and high-dimensional state representations. Our results show orders of magnitude reductions in samples and violations compared to state-of-the-art methods. Lastly, we provide intuition as to the effectiveness of the framework by a detailed analysis of our active metrics and safety constraints.