mlsanityData to 14 Oct 2026

Computationally and Sample Efficient Safe Reinforcement Learning Using Adaptive Conformal Prediction

arXiv 2503.17678 ↗First posted 22 Mar 2025cs.ROcited by 4

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safety is a critical concern in learning-enabled autonomous systems especially when deploying these systems in real-world scenarios. An important challenge is accurately quantifying the uncertainty of unknown models to generate provably safe control policies that facilitate the gathering of informative data, thereby achieving both safe and optimal policies. Additionally, the selection of the data-driven model can significantly impact both the real-time implementation and the uncertainty quantification process. In this paper, we propose a provably sample efficient episodic safe learning framework that remains robust across various model choices with quantified uncertainty for online control tasks. Specifically, we first employ Quadrature Fourier Features (QFF) for kernel function approximation of Gaussian Processes (GPs) to enable efficient approximation of unknown dynamics. Then the Adaptive Conformal Prediction (ACP) is used to quantify the uncertainty from online observations and combined with the Control Barrier Functions (CBF) to characterize the uncertainty-aware safe control constraints under learned dynamics. Finally, an optimism-based exploration strategy is integrated with ACP-based CBFs for safe exploration and near-optimal safe nonlinear control. Theoretical proofs and simulation results are provided to demonstrate the effectiveness and efficiency of the proposed framework.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 31 references that are in the map. First posted on arXiv 22 Mar 2025.

Builds on

24 of 31 references.

References · 24

  1. 2016OpenAI Gymcited by 1,736
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  3. 2017A Tutorial on Thompson Samplingcited by 433
  4. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  5. 2019Control Barrier Functions: Theory and Applicationscited by 372
  6. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  7. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  8. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  9. 2024Reinforcement learningcited by 247
  10. 2022Adaptive Conformal Predictions for Time Seriescited by 161
  11. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 128
  12. 2018Learning-based Model Predictive Control for Safe Explorationcited by 120
12 more
  1. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 90
  2. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 90
  3. 2020Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planningcited by 86
  4. 2017Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Controlcited by 53
  5. 2022Adaptive Conformal Prediction for Motion Planning among Dynamic Agentscited by 53
  6. 2017Safe Learning of Quadrotor Dynamics Using Barrier Certificatescited by 35
  7. 2019On Online Learning in Kernelized Markov Decision Processescited by 35
  8. 2018PG-TS: Improved Thompson Sampling for Logistic Contextual Banditscited by 19
  9. 2019Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificatescited by 13
  10. 2024Safety-Critical Control with Uncertainty Quantification using Adaptive Conformal Predictioncited by 7
  11. 2017Gaussian Process Model Predictive Control of An Unmanned Quadrotorcited by 6
  12. 2022Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functionscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluationcited by 2
  2. 2026Audited Selective Verification for Risk-Controlled N-1 Thermal Contingency Screening under Deployment Shiftcited by 1
  3. 2025Avoiding Death through Fear Intrinsic Conditioningcited by 0
  4. 2026Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  2. 2026Sampling-Based Safe Reinforcement Learning
  3. 2026Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models
  4. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  5. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shifts
  6. 2025Safe and Near-Optimal Control with Online Dynamics Learning
  7. 2025A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems
  8. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization