# A statistical framework for efficient out of distribution detection in deep neural networks

arXiv: https://arxiv.org/abs/2102.12967 · HTML: https://mlsanity.com/paper/2102.12967/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning](https://mlsanity.com/paper/1803.04765/) (2018)
- [Contrastive Training for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2007.05566/) (2020)
- [On the (Statistical) Detection of Adversarial Examples](https://mlsanity.com/paper/1702.06280/) (2017)
- [Hybrid Models for Open Set Recognition](https://mlsanity.com/paper/2003.12506/) (2020)
- [Robust Physical-World Attacks on Deep Learning Models](https://mlsanity.com/paper/1707.08945/) (2017)
- [KS(conf ): A Light-Weight Test if a ConvNet Operates Outside of Its Specifications](https://mlsanity.com/paper/1804.04171/) (2018)
- [Testing the equality of multivariate means when p>n by combining the Hoteling and Simes tests](https://mlsanity.com/paper/1912.10472/) (2019)

## Built on by (most cited first, 28 of 28)

- [RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2209.08590/) (2022)
- [Boosting Out-of-distribution Detection with Typical Features](https://mlsanity.com/paper/2210.04200/) (2022)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI](https://mlsanity.com/paper/2411.02381/) (2024)
- [Conformal link prediction for false discovery rate control](https://mlsanity.com/paper/2306.14693/) (2023)
- [Boosting Out-of-Distribution Detection with Multiple Pre-trained Models](https://mlsanity.com/paper/2212.12720/) (2022)
- [Out-of-Distribution Detection Should Use Conformal Prediction (and Vice-versa?)](https://mlsanity.com/paper/2403.11532/) (2024)
- [Statistical Context Detection for Deep Lifelong Reinforcement Learning](https://mlsanity.com/paper/2405.19047/) (2024)
- [Out-of-Distribution Learning with Human Feedback](https://mlsanity.com/paper/2408.07772/) (2024)
- [Faithfulness Measurable Masked Language Models](https://mlsanity.com/paper/2310.07819/) (2023)
- [Leave-One-Out-, Bootstrap- and Cross-Conformal Anomaly Detectors](https://mlsanity.com/paper/2402.16388/) (2024)
- [Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection](https://mlsanity.com/paper/2408.15580/) (2024)
- [Multiple Testing Framework for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.09522/) (2022)
- [CODiT: Conformal Out-of-Distribution Detection in Time-Series Data](https://mlsanity.com/paper/2207.11769/) (2022)
- [A Functional Data Perspective and Baseline On Multi-Layer Out-of-Distribution Detection](https://mlsanity.com/paper/2306.03522/) (2023)
- [RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection](https://mlsanity.com/paper/2311.13959/) (2023)
- [Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism](https://mlsanity.com/paper/2402.12997/) (2024)
- [A View on Out-of-Distribution Identification from a Statistical Testing Theory Perspective](https://mlsanity.com/paper/2405.03052/) (2024)
- [Combine and Conquer: A Meta-Analysis on Data Shift and Out-of-Distribution Detection](https://mlsanity.com/paper/2406.16045/) (2024)
- [Causal Responder Detection](https://mlsanity.com/paper/2406.17571/) (2024)
- [Constraining Representations Yields Models That Know What They Don't Know](https://mlsanity.com/paper/2208.14488/) (2022)
- [On the Detection of Anomalous or Out-Of-Distribution Data in Vision Models Using Statistical Techniques](https://mlsanity.com/paper/2403.15497/) (2024)
- [Cascading Unknown Detection with Known Classification for Open Set Recognition](https://mlsanity.com/paper/2406.06351/) (2024)
- [DSDE: Using Proportion Estimation to Improve Model Selection for Out-of-Distribution Detection](https://mlsanity.com/paper/2411.01487/) (2024)
- [Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM](https://mlsanity.com/paper/2512.09067/) (2025)
- [Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing](https://mlsanity.com/paper/2605.26429/) (2026)
- [Unified Multi-Layer Subspace Modeling for Cross-Domain OOD Detection](https://mlsanity.com/paper/2609.06785/) (2026)

## Abstract (arXiv)

Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle and unreliable when the test samples are drawn from a dissimilar distribution. This is a major concern for deployment in real-world applications, where such behavior may come at a considerable cost, such as industrial production lines, autonomous vehicles, or healthcare applications. Contributions. We frame Out Of Distribution (OOD) detection in DNNs as a statistical hypothesis testing problem. Tests generated within our proposed framework combine evidence from the entire network. Unlike previous OOD detection heuristics, this framework returns a p-value for each test sample. It is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD for an actual in-distribution sample) for test data. Moreover, this allows to combine several detectors while maintaining the T1E. Building on this framework, we suggest a novel OOD procedure based on low-order statistics. Our method achieves comparable or better results than state-of-the-art methods on well-accepted OOD benchmarks, without retraining the network parameters or assuming prior knowledge on the test distribution -- and at a fraction of the computational cost.