# Model-agnostic out-of-distribution detection using combined statistical tests

arXiv: https://arxiv.org/abs/2203.01097 · HTML: https://mlsanity.com/paper/2203.01097/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 70 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [Input complexity and out-of-distribution detection with likelihood-based generative models](https://mlsanity.com/paper/1909.11480/) (2019)
- [Contrastive Training for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2007.05566/) (2020)
- [Handling Incomplete Heterogeneous Data using VAEs](https://mlsanity.com/paper/1807.03653/) (2018)
- [Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder](https://mlsanity.com/paper/2003.02977/) (2020)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling](https://mlsanity.com/paper/1902.02102/) (2019)
- [Understanding Failures in Out-of-Distribution Detection with Deep Generative Models](https://mlsanity.com/paper/2107.06908/) (2021)
- [EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE](https://mlsanity.com/paper/1809.11142/) (2018)
- [BackPACK: Packing more into backprop](https://mlsanity.com/paper/1912.10985/) (2019)
- [Density of States Estimation for Out-of-Distribution Detection](https://mlsanity.com/paper/2006.09273/) (2020)
- [Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features](https://mlsanity.com/paper/2006.10848/) (2020)
- [not-MIWAE: Deep Generative Modelling with Missing not at Random Data](https://mlsanity.com/paper/2006.12871/) (2020)
- [Hierarchical VAEs Know What They Don't Know](https://mlsanity.com/paper/2102.08248/) (2021)
- [Efficient Per-Example Gradient Computations](https://mlsanity.com/paper/1510.01799/) (2015)
- [Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality](https://mlsanity.com/paper/1906.02994/) (2019)

## Built on by (most cited first, 14 of 14)

- [Unsupervised Out-of-Distribution Detection with Diffusion Inpainting](https://mlsanity.com/paper/2302.10326/) (2023)
- [Boosting Out-of-distribution Detection with Typical Features](https://mlsanity.com/paper/2210.04200/) (2022)
- [Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty](https://mlsanity.com/paper/2212.12641/) (2022)
- [Boosting Out-of-Distribution Detection with Multiple Pre-trained Models](https://mlsanity.com/paper/2212.12720/) (2022)
- [A Unified Data Representation Learning for Non-parametric Two-sample Testing](https://mlsanity.com/paper/2412.00613/) (2024)
- [Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/2411.10701/) (2024)
- [Distribution Guided Active Feature Acquisition](https://mlsanity.com/paper/2410.03915/) (2024)
- [Multiple Testing Framework for Out-of-Distribution Detection](https://mlsanity.com/paper/2206.09522/) (2022)
- [Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection](https://mlsanity.com/paper/2403.01485/) (2024)
- [Out-of-Distribution Detection using BiGAN and MDL](https://mlsanity.com/paper/2206.01851/) (2022)
- [Thinking in Groups: Permutation Tests Reveal Near-Out-of-Distribution](https://mlsanity.com/paper/2403.14058/) (2024)
- [An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Norms](https://mlsanity.com/paper/2603.29466/) (2026)
- [Rethinking Test-time Likelihood: The Likelihood Path Principle and Its Application to OOD Detection](https://mlsanity.com/paper/2401.04933/) (2024)
- [DSDE: Using Proportion Estimation to Improve Model Selection for Out-of-Distribution Detection](https://mlsanity.com/paper/2411.01487/) (2024)

## Abstract (arXiv)

We present simple methods for out-of-distribution detection using a trained generative model. These techniques, based on classical statistical tests, are model-agnostic in the sense that they can be applied to any differentiable generative model. The idea is to combine a classical parametric test (Rao's score test) with the recently introduced typicality test. These two test statistics are both theoretically well-founded and exploit different sources of information based on the likelihood for the typicality test and its gradient for the score test. We show that combining them using Fisher's method overall leads to a more accurate out-of-distribution test. We also discuss the benefits of casting out-of-distribution detection as a statistical testing problem, noting in particular that false positive rate control can be valuable for practical out-of-distribution detection. Despite their simplicity and generality, these methods can be competitive with model-specific out-of-distribution detection algorithms without any assumptions on the out-distribution.