# Unifying Evaluation of Machine Learning Safety Monitors

arXiv: https://arxiv.org/abs/2208.14660 · HTML: https://mlsanity.com/paper/2208.14660/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Misbehaviour Prediction for Autonomous Driving Systems](https://mlsanity.com/paper/1910.04443/) (2019)
- [DOCTOR: A Simple Method for Detecting Misclassification Errors](https://mlsanity.com/paper/2106.02395/) (2021)
- [Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing](https://mlsanity.com/paper/1812.05793/) (2018)
- [Runtime Monitoring Neuron Activation Patterns](https://mlsanity.com/paper/1809.06573/) (2018)
- [Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends](https://mlsanity.com/paper/2101.01364/) (2021)
- [Into the Unknown: Active Monitoring of Neural Networks](https://mlsanity.com/paper/2009.06429/) (2020)
- [Practical Solutions for Machine Learning Safety in Autonomous Vehicles](https://mlsanity.com/paper/1912.09630/) (2019)
- [Benchmarking Safety Monitors for Image Classifiers with Machine Learning](https://mlsanity.com/paper/2110.01232/) (2021)
- [Certifying Emergency Landing for Safe Urban UAV](https://mlsanity.com/paper/2104.14928/) (2021)
- [Fault-Tolerant Perception for Automated Driving A Lightweight Monitoring Approach](https://mlsanity.com/paper/2111.12360/) (2021)
- [Evaluation of Runtime Monitoring for UAV Emergency Landing](https://mlsanity.com/paper/2202.03059/) (2022)

## Built on by (most cited first, 10 of 10)

- [Out-Of-Distribution Detection Is Not All You Need](https://mlsanity.com/paper/2211.16158/) (2022)
- [Combining Cost-Constrained Runtime Monitors for AI Safety](https://mlsanity.com/paper/2507.15886/) (2025)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Can we Defend Against the Unknown? An Empirical Study About Threshold Selection for Neural Network Monitoring](https://mlsanity.com/paper/2405.08654/) (2024)
- [Ensembling Uncertainty Measures to Improve Safety of Black-Box Classifiers](https://mlsanity.com/paper/2308.12065/) (2023)
- [Explaining Unreliable Perception in Automated Driving: A Fuzzy-based Monitoring Approach](https://mlsanity.com/paper/2505.14407/) (2025)
- [Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing](https://mlsanity.com/paper/2604.26411/) (2026)
- [Discriminating Fixture Coverage in Agent-Infrastructure Verification Suites](https://mlsanity.com/paper/2610.02928/) (2026)
- [From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agents](https://mlsanity.com/paper/2610.04575/) (2026)

## Abstract (arXiv)

With the increasing use of Machine Learning (ML) in critical autonomous systems, runtime monitors have been developed to detect prediction errors and keep the system in a safe state during operations. Monitors have been proposed for different applications involving diverse perception tasks and ML models, and specific evaluation procedures and metrics are used for different contexts. This paper introduces three unified safety-oriented metrics, representing the safety benefits of the monitor (Safety Gain), the remaining safety gaps after using it (Residual Hazard), and its negative impact on the system's performance (Availability Cost). To compute these metrics, one requires to define two return functions, representing how a given ML prediction will impact expected future rewards and hazards. Three use-cases (classification, drone landing, and autonomous driving) are used to demonstrate how metrics from the literature can be expressed in terms of the proposed metrics. Experimental results on these examples show how different evaluation choices impact the perceived performance of a monitor. As our formalism requires us to formulate explicit safety assumptions, it allows us to ensure that the evaluation conducted matches the high-level system requirements.