mlsanityData to 14 Oct 2026

Safe exploration in reproducing kernel Hilbert spaces

arXiv 2503.10352 ↗First posted 13 Mar 2025cs.LG cs.SY eess.SYcited by 10

Not on the map. Why?

Abstract (arXiv)

Popular safe Bayesian optimization (BO) algorithms learn control policies for safety-critical systems in unknown environments. However, most algorithms make a smoothness assumption, which is encoded by a known bounded norm in a reproducing kernel Hilbert space (RKHS). The RKHS is a potentially infinite-dimensional space, and it remains unclear how to reliably obtain the RKHS norm of an unknown function. In this work, we propose a safe BO algorithm capable of estimating the RKHS norm from data. We provide statistical guarantees on the RKHS norm estimation, integrate the estimated RKHS norm into existing confidence intervals and show that we retain theoretical guarantees, and prove safety of the resulting safe BO algorithm. We apply our algorithm to safely optimize reinforcement learning policies on physics simulators and on a real inverted pendulum, demonstrating improved performance, safety, and scalability compared to the state-of-the-art.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 21 of its 58 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

21 of 58 references.

References · 21

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2016OpenAI Gymcited by 1,736
  4. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
  5. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  6. 2018Stagewise Safe Bayesian Optimization with Gaussian Processescited by 79
  7. 2015Safe Controller Optimization for Quadrotors with Gaussian Processescited by 78
  8. 2019No-Regret Bayesian Optimization with Unknown Hyperparameterscited by 51
  9. 2021Practical and Rigorous Uncertainty Bounds for Gaussian Process Regressioncited by 49
  10. 2016Automatic LQR Tuning Based on Gaussian Process Global Optimizationcited by 30
  11. 2021GoSafe: Globally Optimal Safe Robot Learningcited by 27
  12. 2020Deterministic error bounds for kernel-based learning techniques under bounded noisecited by 24
9 more
  1. 2022GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systemscited by 17
  2. 2020On the exact feasibility of convex scenario programs with discarded constraintscited by 15
  3. 2024On Safety in Safe Bayesian Optimizationcited by 15
  4. 2017Deep Kernels for Optimizing Locomotion Controllerscited by 14
  5. 2021Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approachcited by 14
  6. 2023Automatic nonlinear MPC approximation with closed-loop guaranteescited by 12
  7. 2020Learning-based Symbolic Abstractions for Nonlinear Control Systemscited by 11
  8. 2024PACSBO: Probably approximately correct safe Bayesian optimizationcited by 8
  9. 2022Orthonormal Expansions for Translation-Invariant Kernelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025Safe and Near-Optimal Control with Online Dynamics Learningcited by 1
  2. 2025Safe Bayesian optimization across noise models via scenario programmingcited by 1
  3. 2026Computationally lightweight classifiers with frequentist bounds on predictionscited by 1
  4. 2026Safe learning-based control via function-based uncertainty quantificationcited by 1
  5. 2025Towards safe control parameter tuning in distributed multi-agent systemscited by 0
  6. 2026Asynchronous Cooperative Online Learning for Multi-Robot Control under Computational Delayscited by 0
  7. 2026Event-triggered Control and Online Learning for Networked Systems under Computational Delayscited by 0
  8. 2026Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guidecited by 0
  9. 2026COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regressioncited by 0
  10. 2026Computationally efficient safe exploration in reinforcement learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Sampling-Based Safe Reinforcement Learning
  2. 2025Sampling Complexity of TD and PPO in RKHS
  3. 2026Data-Driven Synthesis of Probabilistic Controlled Invariant Sets for Linear MDPs
  4. 2026Kernel-Based Safe Exploration in Deep Reinforcement Learning
  5. 2025Policy Newton Algorithm in Reproducing Kernel Hilbert Space
  6. 2024Physics-informed RL for Maximal Safety Probability Estimation
  7. 2025Tailoring Reproducing Kernels for Optimal Control via Policy Iteration
  8. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems