# Safe Learning and Optimization Techniques: Towards a Survey of the State of the Art

arXiv: https://arxiv.org/abs/2101.09505 · HTML: https://mlsanity.com/paper/2101.09505/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 38 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 7 of 7)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [Safe Exploration in Finite Markov Decision Processes with Gaussian Processes](https://mlsanity.com/paper/1606.04753/) (2016)
- [Safe Exploration for Interactive Machine Learning](https://mlsanity.com/paper/1910.13726/) (2019)
- [Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models](https://mlsanity.com/paper/1904.01068/) (2019)

## Built on by (most cited first, 16 of 16)

- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [A Survey of Constraint Formulations in Safe Reinforcement Learning](https://mlsanity.com/paper/2402.02025/) (2024)
- [Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes](https://mlsanity.com/paper/2110.11383/) (2021)
- [On Safety in Safe Bayesian Optimization](https://mlsanity.com/paper/2403.12948/) (2024)
- [Falsification of Learning-Based Controllers through Multi-Fidelity Bayesian Optimization](https://mlsanity.com/paper/2212.14118/) (2022)
- [Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art](https://mlsanity.com/paper/2601.05152/) (2026)
- [Heterogeneous Objectives: State-of-the-Art and Future Research](https://mlsanity.com/paper/2103.15546/) (2021)
- [Modeling Risk in Reinforcement Learning: A Literature Mapping](https://mlsanity.com/paper/2312.05231/) (2023)
- [Physics-informed RL for Maximal Safety Probability Estimation](https://mlsanity.com/paper/2403.16391/) (2024)
- [Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learning](https://mlsanity.com/paper/2511.04147/) (2025)
- [Are Evolutionary Algorithms Safe Optimizers?](https://mlsanity.com/paper/2203.12622/) (2022)
- [Learning Disturbances Online for Risk-Aware Control: Risk-Aware Flight with Less Than One Minute of Data](https://mlsanity.com/paper/2212.06253/) (2022)
- [Probabilistic Counterexample Guidance for Safer Reinforcement Learning (Extended Version)](https://mlsanity.com/paper/2307.04927/) (2023)
- [Optimizing Falsification for Learning-Based Control Systems: A Multi-Fidelity Bayesian Approach](https://mlsanity.com/paper/2409.08097/) (2024)
- [Counterfactual Credit Guided Bayesian Optimization](https://mlsanity.com/paper/2510.04676/) (2025)

## Abstract (arXiv)

Safe learning and optimization deals with learning and optimization problems that avoid, as much as possible, the evaluation of non-safe input points, which are solutions, policies, or strategies that cause an irrecoverable loss (e.g., breakage of a machine or equipment, or life threat). Although a comprehensive survey of safe reinforcement learning algorithms was published in 2015, a number of new algorithms have been proposed thereafter, and related works in active learning and in optimization were not considered. This paper reviews those algorithms from a number of domains including reinforcement learning, Gaussian process regression and classification, evolutionary algorithms, and active learning. We provide the fundamental concepts on which the reviewed algorithms are based and a characterization of the individual algorithms. We conclude by explaining how the algorithms are connected and suggestions for future research.