# Learning Robot Safety from Sparse Human Feedback using Conformal Prediction

arXiv: https://arxiv.org/abs/2501.04823 · HTML: https://mlsanity.com/paper/2501.04823/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2025.

## Built on by (most cited first, 2 of 2)

- [Conformal Safety Monitoring for Flight Testing: A Case Study in Data-Driven Safety Learning](https://mlsanity.com/paper/2511.20811/) (2025)
- [Certified Safety Curation: Distribution-Free Guarantees for Safe Offline Reinforcement Learning](https://mlsanity.com/paper/2609.12014/) (2026)

## Abstract (arXiv)

Ensuring robot safety can be challenging; user-defined constraints can miss edge cases, policies can become unsafe even when trained from safe data, and safety can be subjective. Thus, we learn about robot safety by showing policy trajectories to a human who flags unsafe behavior. From this binary feedback, we use the statistical method of conformal prediction to identify a region of states, potentially in learned latent space, guaranteed to contain a user-specified fraction of future policy errors. Our method is sample-efficient, as it builds on nearest neighbor classification and avoids withholding data as is common with conformal prediction. By alerting if the robot reaches the suspected unsafe region, we obtain a warning system that mimics the human's safety preferences with guaranteed miss rate. From video labeling, our system can detect when a quadcopter visuomotor policy will fail to steer through a designated gate. We present an approach for policy improvement by avoiding the suspected unsafe region. With it we improve a model predictive controller's safety, as shown in experimental testing with 30 quadcopter flights across 6 navigation tasks. Code and videos are provided.