mlsanityData to 14 Oct 2026

Computationally efficient safe exploration in reinforcement learning

arXiv 2609.22919 ↗First posted 19 Sep 2026cs.LG cs.RO cs.SYcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Reinforcement learning in real-life applications requires safety guarantees during exploration. Typical reinforcement learning algorithms do not provide such guarantees, and many modifications that do rely on Gaussian processes (GPs), which have a large computational cost. We propose a computationally lightweight algorithm based on the Nadaraya-Watson estimator that safely explores and optimizes constrained Markov decision processes (MDPs). Our algorithm, textscCoLSafe-MDP, uses an estimator that scales in constant-time with bounds on the estimates, a significant improvement from its GP-based counterparts that scale cubically with the number of data points. We then evaluate its performance in a grid-based environment and on observational Martian terrain data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 34 references that are in the map. First posted on arXiv 19 Sep 2026.

Builds on

19 of 34 references.

References · 19

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  3. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  4. 2017On Kernelized Multi-armed Banditscited by 352
  5. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  6. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  7. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  8. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 75
  9. 2014Lipschitz Bandits: Regret Lower Bounds and Optimal Algorithmscited by 60
  10. 2019Safe Exploration for Interactive Machine Learningcited by 51
  11. 2018Probabilistically Safe Robot Planning with Confidence-Based Human Predictionscited by 28
  12. 2023Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPscited by 28
7 more
  1. 2024On Safety in Safe Bayesian Optimizationcited by 15
  2. 2023A safe exploration approach to constrained Markov decision processescited by 10
  3. 2025Safe exploration in reproducing kernel Hilbert spacescited by 10
  4. 2023A computationally lightweight safe learning algorithmcited by 3
  5. 2025Safety and optimality in learning-based control at low computational costcited by 2
  6. 2026Computationally lightweight classifiers with frequentist bounds on predictionscited by 1
  7. 2026Safe learning-based control via function-based uncertainty quantificationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Safe Reinforcement Learning for Constrained Markov Decision Processes with Stochastic Stopping Time
  2. 2024Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
  3. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization
  4. 2025Ensuring Safety in an Uncertain Environment: Constrained MDPs via Stochastic Thresholds
  5. 2026Sampling-Based Safe Reinforcement Learning
  6. 2026Safe Exploration via Policy Priors
  7. 2024Model-Free Active Exploration in Reinforcement Learning
  8. 2025Model-Based Exploration in Monitored Markov Decision Processes