# Performance Analysis of Out-of-Distribution Detection on Trained Neural Networks

arXiv: https://arxiv.org/abs/2204.12378 · HTML: https://mlsanity.com/paper/2204.12378/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [DLFuzz: Differential Fuzzing Testing of Deep Learning Systems](https://mlsanity.com/paper/1808.09413/) (2018)
- [Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry](https://mlsanity.com/paper/1812.05389/) (2018)
- [An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software](https://mlsanity.com/paper/1709.02435/) (2017)
- [Improved Robustness and Safety for Autonomous Vehicle Control with Adversarial Reinforcement Learning](https://mlsanity.com/paper/1903.03642/) (2019)
- [Towards Structured Evaluation of Deep Neural Network Supervisors](https://mlsanity.com/paper/1903.01263/) (2019)
- [Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Driving](https://mlsanity.com/paper/1903.03438/) (2019)
- [Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks](https://mlsanity.com/paper/2103.15580/) (2021)

## Built on by (most cited first, 7 of 7)

- [Generalization in Neural Networks: A Broad Survey](https://mlsanity.com/paper/2209.01610/) (2022)
- [Evaluation of Out-of-Distribution Detection Performance on Autonomous Driving Datasets](https://mlsanity.com/paper/2401.17013/) (2024)
- [A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement](https://mlsanity.com/paper/2411.04090/) (2024)
- [Hybrid Video Anomaly Detection for Anomalous Scenarios in Autonomous Driving](https://mlsanity.com/paper/2406.06423/) (2024)
- [Trust Calibration in IDEs: Paving the Way for Widespread Adoption of AI Refactoring](https://mlsanity.com/paper/2412.15948/) (2024)
- [Borch: A Deep Universal Probabilistic Programming Language](https://mlsanity.com/paper/2209.06168/) (2022)
- [Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins](https://mlsanity.com/paper/2605.21493/) (2026)

## Abstract (arXiv)

Several areas have been improved with Deep Learning during the past years. Implementing Deep Neural Networks (DNN) for non-safety related applications have shown remarkable achievements over the past years; however, for using DNNs in safety critical applications, we are missing approaches for verifying the robustness of such models. A common challenge for DNNs occurs when exposed to out-of-distribution samples that are outside of the scope of a DNN, but which result in high confidence outputs despite no prior knowledge of such input. In this paper, we analyze three methods that separate between in- and out-of-distribution data, called supervisors, on four well-known DNN architectures. We find that the outlier detection performance improves with the quality of the model. We also analyse the performance of the particular supervisors during the training procedure by applying the supervisor at a predefined interval to investigate its performance as the training proceeds. We observe that understanding the relationship between training results and supervisor performance is crucial to improve the model's robustness and to indicate, what input samples require further measures to improve the robustness of a DNN. In addition, our work paves the road towards an instrument for safety argumentation for safety critical applications. This paper is an extended version of our previous work presented at 2019 SEAA (cf. [1]); here, we elaborate on the used metrics, add an additional supervisor and test them on two additional datasets.