# Benchmarking Safety Monitors for Image Classifiers with Machine Learning

arXiv: https://arxiv.org/abs/2110.01232 · HTML: https://mlsanity.com/paper/2110.01232/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 4 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance](https://mlsanity.com/paper/1812.02765/) (2018)
- [Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing](https://mlsanity.com/paper/2007.02787/) (2020)
- [Outside the Box: Abstraction-Based Monitoring of Neural Networks](https://mlsanity.com/paper/1911.09032/) (2019)
- [Runtime Monitoring Neuron Activation Patterns](https://mlsanity.com/paper/1809.06573/) (2018)
- [Deep Transfer Learning for Multiple Class Novelty Detection](https://mlsanity.com/paper/1903.02196/) (2019)
- [Compositional Falsification of Cyber-Physical Systems with Machine Learning Components](https://mlsanity.com/paper/1703.00978/) (2017)
- [Boosting Adversarial Attacks with Momentum](https://mlsanity.com/paper/1710.06081/) (2017)
- [Model Assertions for Monitoring and Improving ML Models](https://mlsanity.com/paper/2003.01668/) (2020)
- [Towards Structured Evaluation of Deep Neural Network Supervisors](https://mlsanity.com/paper/1903.01263/) (2019)
- [Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring](https://mlsanity.com/paper/2102.00902/) (2021)
- [Input Validation for Neural Networks via Runtime Local Robustness Verification](https://mlsanity.com/paper/2002.03339/) (2020)

## Built on by (most cited first, 14 of 14)

- [Out-Of-Distribution Detection Is Not All You Need](https://mlsanity.com/paper/2211.16158/) (2022)
- [Unifying Evaluation of Machine Learning Safety Monitors](https://mlsanity.com/paper/2208.14660/) (2022)
- [SMARLA: A Safety Monitoring Approach for Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2308.02594/) (2023)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Evaluation of Runtime Monitoring for UAV Emergency Landing](https://mlsanity.com/paper/2202.03059/) (2022)
- [Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications](https://mlsanity.com/paper/2402.08208/) (2024)
- [Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines](https://mlsanity.com/paper/2212.07118/) (2022)
- [Safety Monitoring of Machine Learning Perception Functions: a Survey](https://mlsanity.com/paper/2412.06869/) (2024)
- [Monitizer: Automating Design and Evaluation of Neural Network Monitors](https://mlsanity.com/paper/2405.10350/) (2024)
- [CheapET-3: Cost-Efficient Use of Remote DNN Models](https://mlsanity.com/paper/2208.11552/) (2022)
- [A New Perspective On AI Safety Through Control Theory Methodologies](https://mlsanity.com/paper/2506.23703/) (2025)
- [Attention Masks Help Adversarial Attacks to Bypass Safety Detectors](https://mlsanity.com/paper/2411.04772/) (2024)
- [Explaining Unreliable Perception in Automated Driving: A Fuzzy-based Monitoring Approach](https://mlsanity.com/paper/2505.14407/) (2025)
- [Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing](https://mlsanity.com/paper/2604.26411/) (2026)

## Abstract (arXiv)

High-accurate machine learning (ML) image classifiers cannot guarantee that they will not fail at operation. Thus, their deployment in safety-critical applications such as autonomous vehicles is still an open issue. The use of fault tolerance mechanisms such as safety monitors is a promising direction to keep the system in a safe state despite errors of the ML classifier. As the prediction from the ML is the core information directly impacting safety, many works are focusing on monitoring the ML model itself. Checking the efficiency of such monitors in the context of safety-critical applications is thus a significant challenge. Therefore, this paper aims at establishing a baseline framework for benchmarking monitors for ML image classifiers. Furthermore, we propose a framework covering the entire pipeline, from data generation to evaluation. Our approach measures monitor performance with a broader set of metrics than usually proposed in the literature. Moreover, we benchmark three different monitor approaches in 79 benchmark datasets containing five categories of out-of-distribution data for image classifiers: class novelty, noise, anomalies, distributional shifts, and adversarial attacks. Our results indicate that these monitors are no more accurate than a random monitor. We also release the code of all experiments for reproducibility.