# Safe exploration in reproducing kernel Hilbert spaces

arXiv: https://arxiv.org/abs/2503.10352 · HTML: https://mlsanity.com/paper/2503.10352/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 58 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [Exactly Computing the Local Lipschitz Constant of ReLU Networks](https://mlsanity.com/paper/2003.01219/) (2020)
- [Stagewise Safe Bayesian Optimization with Gaussian Processes](https://mlsanity.com/paper/1806.07555/) (2018)
- [Safe Controller Optimization for Quadrotors with Gaussian Processes](https://mlsanity.com/paper/1509.01066/) (2015)
- [No-Regret Bayesian Optimization with Unknown Hyperparameters](https://mlsanity.com/paper/1901.03357/) (2019)
- [Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression](https://mlsanity.com/paper/2105.02796/) (2021)
- [Automatic LQR Tuning Based on Gaussian Process Global Optimization](https://mlsanity.com/paper/1605.01950/) (2016)
- [GoSafe: Globally Optimal Safe Robot Learning](https://mlsanity.com/paper/2105.13281/) (2021)
- [Deterministic error bounds for kernel-based learning techniques under bounded noise](https://mlsanity.com/paper/2008.04005/) (2020)
- [GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems](https://mlsanity.com/paper/2201.09562/) (2022)
- [On the exact feasibility of convex scenario programs with discarded constraints](https://mlsanity.com/paper/2003.13433/) (2020)
- [On Safety in Safe Bayesian Optimization](https://mlsanity.com/paper/2403.12948/) (2024)
- [Deep Kernels for Optimizing Locomotion Controllers](https://mlsanity.com/paper/1707.09062/) (2017)
- [Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approach](https://mlsanity.com/paper/2104.09582/) (2021)
- [Automatic nonlinear MPC approximation with closed-loop guarantees](https://mlsanity.com/paper/2312.10199/) (2023)
- [Learning-based Symbolic Abstractions for Nonlinear Control Systems](https://mlsanity.com/paper/2004.01879/) (2020)
- [PACSBO: Probably approximately correct safe Bayesian optimization](https://mlsanity.com/paper/2409.01163/) (2024)
- [Orthonormal Expansions for Translation-Invariant Kernels](https://mlsanity.com/paper/2206.08648/) (2022)

## Built on by (most cited first, 10 of 10)

- [Safe and Near-Optimal Control with Online Dynamics Learning](https://mlsanity.com/paper/2509.16650/) (2025)
- [Safe Bayesian optimization across noise models via scenario programming](https://mlsanity.com/paper/2512.11580/) (2025)
- [Computationally lightweight classifiers with frequentist bounds on predictions](https://mlsanity.com/paper/2603.22128/) (2026)
- [Safe learning-based control via function-based uncertainty quantification](https://mlsanity.com/paper/2604.01173/) (2026)
- [Towards safe control parameter tuning in distributed multi-agent systems](https://mlsanity.com/paper/2508.13608/) (2025)
- [Asynchronous Cooperative Online Learning for Multi-Robot Control under Computational Delays](https://mlsanity.com/paper/2608.29562/) (2026)
- [Event-triggered Control and Online Learning for Networked Systems under Computational Delays](https://mlsanity.com/paper/2608.29576/) (2026)
- [Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guide](https://mlsanity.com/paper/2608.30051/) (2026)
- [COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regression](https://mlsanity.com/paper/2609.20598/) (2026)
- [Computationally efficient safe exploration in reinforcement learning](https://mlsanity.com/paper/2609.22919/) (2026)

## Abstract (arXiv)

Popular safe Bayesian optimization (BO) algorithms learn control policies for safety-critical systems in unknown environments. However, most algorithms make a smoothness assumption, which is encoded by a known bounded norm in a reproducing kernel Hilbert space (RKHS). The RKHS is a potentially infinite-dimensional space, and it remains unclear how to reliably obtain the RKHS norm of an unknown function. In this work, we propose a safe BO algorithm capable of estimating the RKHS norm from data. We provide statistical guarantees on the RKHS norm estimation, integrate the estimated RKHS norm into existing confidence intervals and show that we retain theoretical guarantees, and prove safety of the resulting safe BO algorithm. We apply our algorithm to safely optimize reinforcement learning policies on physics simulators and on a real inverted pendulum, demonstrating improved performance, safety, and scalability compared to the state-of-the-art.