# Evaluation of Runtime Monitoring for UAV Emergency Landing

arXiv: https://arxiv.org/abs/2202.03059 · HTML: https://mlsanity.com/paper/2202.03059/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 12 of 12)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Misbehaviour Prediction for Autonomous Driving Systems](https://mlsanity.com/paper/1910.04443/) (2019)
- [Runtime Monitoring Neuron Activation Patterns](https://mlsanity.com/paper/1809.06573/) (2018)
- [Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends](https://mlsanity.com/paper/2101.01364/) (2021)
- [Towards Structured Evaluation of Deep Neural Network Supervisors](https://mlsanity.com/paper/1903.01263/) (2019)
- [Benchmarking Safety Monitors for Image Classifiers with Machine Learning](https://mlsanity.com/paper/2110.01232/) (2021)
- [Certifying Emergency Landing for Safe Urban UAV](https://mlsanity.com/paper/2104.14928/) (2021)
- [Robust Autonomous Landing of UAV in Non-Cooperative Environments based on Dynamic Time Camera-LiDAR Fusion](https://mlsanity.com/paper/2011.13761/) (2020)
- [Ensemble Transfer Learning for Emergency Landing Field Identification on Moderate Resource Heterogeneous Kubernetes Cluster](https://mlsanity.com/paper/2006.14887/) (2020)

## Built on by (most cited first, 4 of 4)

- [Out-Of-Distribution Detection Is Not All You Need](https://mlsanity.com/paper/2211.16158/) (2022)
- [Unifying Evaluation of Machine Learning Safety Monitors](https://mlsanity.com/paper/2208.14660/) (2022)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Can we Defend Against the Unknown? An Empirical Study About Threshold Selection for Neural Network Monitoring](https://mlsanity.com/paper/2405.08654/) (2024)

## Abstract (arXiv)

To certify UAV operations in populated areas, risk mitigation strategies -- such as Emergency Landing (EL) -- must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using on-board sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) approaches within the context of a real-world critical system. A new evaluation methodology is introduced, and applied to assess the practical safety benefits of three MLRM mechanisms. The proposed approach is compared to a default mitigation strategy (open a parachute when a failure is detected), and appears to be much safer.