# Safe reinforcement learning in uncertain contexts

arXiv: https://arxiv.org/abs/2401.05876 · HTML: https://mlsanity.com/paper/2401.05876/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics](https://mlsanity.com/paper/1602.04450/) (2016)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Distribution-free binary classification: prediction sets, confidence intervals and calibration](https://mlsanity.com/paper/2006.10564/) (2020)
- [Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression](https://mlsanity.com/paper/2105.02796/) (2021)
- [Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification](https://mlsanity.com/paper/2102.07856/) (2021)
- [Sequence Model Imitation Learning with Unobserved Contexts](https://mlsanity.com/paper/2208.02225/) (2022)
- [Deterministic error bounds for kernel-based learning techniques under bounded noise](https://mlsanity.com/paper/2008.04005/) (2020)
- [Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approach](https://mlsanity.com/paper/2104.09582/) (2021)
- [Closed-form Expressions for Maximum Mean Discrepancy with Applications to Wasserstein Auto-Encoders](https://mlsanity.com/paper/1901.03227/) (2019)
- [Safe and Efficient Model-free Adaptive Control via Bayesian Optimization](https://mlsanity.com/paper/2101.07825/) (2021)
- [A Kernel Two-sample Test for Dynamical Systems](https://mlsanity.com/paper/2004.11098/) (2020)
- [Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models](https://mlsanity.com/paper/2102.11394/) (2021)
- [Hyperparameter Learning for Conditional Kernel Mean Embeddings with Rademacher Complexity Bounds](https://mlsanity.com/paper/1809.00175/) (2018)
- [Learning Fast and Precise Pixel-to-Torque Control](https://mlsanity.com/paper/2208.02315/) (2022)

## Built on by (most cited first, 1 of 1)

- [Computationally lightweight classifiers with frequentist bounds on predictions](https://mlsanity.com/paper/2603.22128/) (2026)

## Abstract (arXiv)

When deploying machine learning algorithms in the real world, guaranteeing safety is an essential asset. Existing safe learning approaches typically consider continuous variables, i.e., regression tasks. However, in practice, robotic systems are also subject to discrete, external environmental changes, e.g., having to carry objects of certain weights or operating on frozen, wet, or dry surfaces. Such influences can be modeled as discrete context variables. In the existing literature, such contexts are, if considered, mostly assumed to be known. In this work, we drop this assumption and show how we can perform safe learning when we cannot directly measure the context variables. To achieve this, we derive frequentist guarantees for multi-class classification, allowing us to estimate the current context from measurements. Further, we propose an approach for identifying contexts through experiments. We discuss under which conditions we can retain theoretical guarantees and demonstrate the applicability of our algorithm on a Furuta pendulum with camera measurements of different weights that serve as contexts.