# Computationally and Sample Efficient Safe Reinforcement Learning Using Adaptive Conformal Prediction

arXiv: https://arxiv.org/abs/2503.17678 · HTML: https://mlsanity.com/paper/2503.17678/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 24 of 24)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Adaptive Conformal Predictions for Time Series](https://mlsanity.com/paper/2202.07282/) (2022)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Learning-based Model Predictive Control for Safe Exploration](https://mlsanity.com/paper/1803.08287/) (2018)
- [Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving](https://mlsanity.com/paper/1707.02342/) (2017)
- [Information Theoretic Regret Bounds for Online Nonlinear Control](https://mlsanity.com/paper/2006.12466/) (2020)
- [Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning](https://mlsanity.com/paper/2006.08684/) (2020)
- [Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control](https://mlsanity.com/paper/1706.06491/) (2017)
- [Adaptive Conformal Prediction for Motion Planning among Dynamic Agents](https://mlsanity.com/paper/2212.00278/) (2022)
- [Safe Learning of Quadrotor Dynamics Using Barrier Certificates](https://mlsanity.com/paper/1710.05472/) (2017)
- [On Online Learning in Kernelized Markov Decision Processes](https://mlsanity.com/paper/1911.01871/) (2019)
- [PG-TS: Improved Thompson Sampling for Logistic Contextual Bandits](https://mlsanity.com/paper/1805.07458/) (2018)
- [Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificates](https://mlsanity.com/paper/1912.09957/) (2019)
- [Safety-Critical Control with Uncertainty Quantification using Adaptive Conformal Prediction](https://mlsanity.com/paper/2407.03569/) (2024)
- [Gaussian Process Model Predictive Control of An Unmanned Quadrotor](https://mlsanity.com/paper/1707.04515/) (2017)
- [Sample-efficient Safe Learning for Online Nonlinear Control with Control Barrier Functions](https://mlsanity.com/paper/2207.14419/) (2022)

## Built on by (most cited first, 4 of 4)

- [Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluation](https://mlsanity.com/paper/2510.26026/) (2025)
- [Audited Selective Verification for Risk-Controlled N-1 Thermal Contingency Screening under Deployment Shift](https://mlsanity.com/paper/2607.13221/) (2026)
- [Avoiding Death through Fear Intrinsic Conditioning](https://mlsanity.com/paper/2506.05529/) (2025)
- [Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximation](https://mlsanity.com/paper/2607.16501/) (2026)

## Abstract (arXiv)

Safety is a critical concern in learning-enabled autonomous systems especially when deploying these systems in real-world scenarios. An important challenge is accurately quantifying the uncertainty of unknown models to generate provably safe control policies that facilitate the gathering of informative data, thereby achieving both safe and optimal policies. Additionally, the selection of the data-driven model can significantly impact both the real-time implementation and the uncertainty quantification process. In this paper, we propose a provably sample efficient episodic safe learning framework that remains robust across various model choices with quantified uncertainty for online control tasks. Specifically, we first employ Quadrature Fourier Features (QFF) for kernel function approximation of Gaussian Processes (GPs) to enable efficient approximation of unknown dynamics. Then the Adaptive Conformal Prediction (ACP) is used to quantify the uncertainty from online observations and combined with the Control Barrier Functions (CBF) to characterize the uncertainty-aware safe control constraints under learned dynamics. Finally, an optimism-based exploration strategy is integrated with ACP-based CBFs for safe exploration and near-optimal safe nonlinear control. Theoretical proofs and simulation results are provided to demonstrate the effectiveness and efficiency of the proposed framework.