# GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems

arXiv: https://arxiv.org/abs/2201.09562 · HTML: https://mlsanity.com/paper/2201.09562/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 62 references that are in the map.
- First posted on arXiv 24 Jan 2022.

## Builds on (most cited first, 30 of 32)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [A Lyapunov-based Approach to Safe Reinforcement Learning](https://mlsanity.com/paper/1805.07708/) (2018)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Learning to be Safe: Deep RL with a Safety Critic](https://mlsanity.com/paper/2010.14603/) (2020)
- [Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1403.5607/) (2014)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces](https://mlsanity.com/paper/1902.03229/) (2019)
- [No-Regret Bayesian Optimization with Unknown Hyperparameters](https://mlsanity.com/paper/1901.03357/) (2019)
- [A General Framework for Constrained Bayesian Optimization using Information-based Search](https://mlsanity.com/paper/1511.09422/) (2015)
- [Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression](https://mlsanity.com/paper/2105.02796/) (2021)
- [Automatic LQR Tuning Based on Gaussian Process Global Optimization](https://mlsanity.com/paper/1605.01950/) (2016)
- [GoSafe: Globally Optimal Safe Robot Learning](https://mlsanity.com/paper/2105.13281/) (2021)
- [Safe Contextual Bayesian Optimization for Sustainable Room Temperature PID Control Tuning](https://mlsanity.com/paper/1906.12086/) (2019)
- [Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1910.13399/) (2019)
- [Robot Learning with Crash Constraints](https://mlsanity.com/paper/2010.08669/) (2020)
- [Deep Kernels for Optimizing Locomotion Controllers](https://mlsanity.com/paper/1707.09062/) (2017)

## Built on by (most cited first, 17 of 17)

- [MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization](https://mlsanity.com/paper/2412.12098/) (2024)
- [ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning](https://mlsanity.com/paper/2410.09486/) (2024)
- [Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction](https://mlsanity.com/paper/2209.13308/) (2022)
- [On Safety in Safe Bayesian Optimization](https://mlsanity.com/paper/2403.12948/) (2024)
- [Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications](https://mlsanity.com/paper/2404.09080/) (2024)
- [Safe exploration in reproducing kernel Hilbert spaces](https://mlsanity.com/paper/2503.10352/) (2025)
- [Tuning Legged Locomotion Controllers via Safe Bayesian Optimization](https://mlsanity.com/paper/2306.07092/) (2023)
- [PACSBO: Probably approximately correct safe Bayesian optimization](https://mlsanity.com/paper/2409.01163/) (2024)
- [Safe Exploration via Policy Priors](https://mlsanity.com/paper/2601.19612/) (2026)
- [Safe Exploration Using Bayesian World Models and Log-Barrier Optimization](https://mlsanity.com/paper/2405.05890/) (2024)
- [A computationally lightweight safe learning algorithm](https://mlsanity.com/paper/2309.03672/) (2023)
- [Safety and optimality in learning-based control at low computational cost](https://mlsanity.com/paper/2505.08026/) (2025)
- [No-Regret Algorithms for Safe Bayesian Optimization with Monotonicity Constraints](https://mlsanity.com/paper/2406.03264/) (2024)
- [Safe Bayesian optimization across noise models via scenario programming](https://mlsanity.com/paper/2512.11580/) (2025)
- [Safely Learning Controlled Stochastic Dynamics](https://mlsanity.com/paper/2506.02754/) (2025)
- [Simulation Priors for Data-Efficient Deep Learning](https://mlsanity.com/paper/2509.05732/) (2025)
- [Foundations of Reinforcement Learning and Control:Connections and New Perspectives](https://mlsanity.com/paper/2608.02433/) (2026)

## Abstract (arXiv)

Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage. Most existing model-free learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. A notable exception is the GoSafe algorithm, which, unfortunately, cannot handle high-dimensional systems and hence cannot be applied to most real-world dynamical systems. This work proposes GoSafeOpt as the first algorithm that can safely discover globally optimal policies for high-dimensional systems while giving safety and optimality guarantees. We demonstrate the superiority of GoSafeOpt over competing model-free safe learning methods on a robot arm that would be prohibitive for GoSafe.