# Conformal Safety Monitoring for Flight Testing: A Case Study in Data-Driven Safety Learning

arXiv: https://arxiv.org/abs/2511.20811 · HTML: https://mlsanity.com/paper/2511.20811/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Safe Planning in Dynamic Environments using Conformal Prediction](https://mlsanity.com/paper/2210.10254/) (2022)
- [Adaptive Conformal Prediction for Motion Planning among Dynamic Agents](https://mlsanity.com/paper/2212.00278/) (2022)
- [A System-Level View on Out-of-Distribution Data in Robotics](https://mlsanity.com/paper/2212.14020/) (2022)
- [Conformal Prediction Regions for Time Series using Linear Complementarity Programming](https://mlsanity.com/paper/2304.01075/) (2023)
- [Multi-Agent Reachability Calibration with Conformal Prediction](https://mlsanity.com/paper/2304.00432/) (2023)
- [Conformal Predictive Safety Filter for RL Controllers in Dynamic Environments](https://mlsanity.com/paper/2306.02551/) (2023)
- [Closing the Loop on Runtime Monitors with Fallback-Safe MPC](https://mlsanity.com/paper/2309.08603/) (2023)
- [Multi-Modal Conformal Prediction Regions with Simple Structures by Optimizing Convex Shape Templates](https://mlsanity.com/paper/2312.07434/) (2023)
- [Learning Robot Safety from Sparse Human Feedback using Conformal Prediction](https://mlsanity.com/paper/2501.04823/) (2025)

## Built on by (most cited first, 1 of 1)

- [How VLAs Fail Differently: Black-Box Action Monitoring Reveals Architecture-Specific Failure Signatures](https://mlsanity.com/paper/2605.28726/) (2026)

## Abstract (arXiv)

We develop a data-driven approach for runtime safety monitoring in flight testing, where pilots perform maneuvers on aircraft with uncertain parameters. Because safety violations can arise unexpectedly as a result of these uncertainties, pilots need clear, preemptive criteria to abort the maneuver in advance of safety violation. To solve this problem, we use offline stochastic trajectory simulation to learn a calibrated statistical model of the short-term safety risk facing pilots. We use flight testing as a motivating example for data-driven learning/monitoring of safety due to its inherent safety risk, uncertainty, and human-interaction. However, our approach consists of three broadly-applicable components: a model to predict future state from recent observations, a nearest neighbor model to classify the safety of the predicted state, and classifier calibration via conformal prediction. We evaluate our method on a flight dynamics model with uncertain parameters, demonstrating its ability to reliably identify unsafe scenarios, match theoretical guarantees, and outperform baseline approaches in preemptive classification of risk.