# A Geometric Explanation of the Likelihood OOD Detection Paradox

arXiv: https://arxiv.org/abs/2403.18910 · HTML: https://mlsanity.com/paper/2403.18910/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 87 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 30 of 57)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift](https://mlsanity.com/paper/1810.11953/) (2018)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [Input complexity and out-of-distribution detection with likelihood-based generative models](https://mlsanity.com/paper/1909.11480/) (2019)
- [A Theory of Generative ConvNet](https://mlsanity.com/paper/1602.03264/) (2016)
- [Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder](https://mlsanity.com/paper/2003.02977/) (2020)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)

## Built on by (most cited first, 22 of 22)

- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models](https://mlsanity.com/paper/2406.03537/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Losing dimensions: Geometric memorization in generative diffusion](https://mlsanity.com/paper/2410.08727/) (2024)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models](https://mlsanity.com/paper/2408.08307/) (2024)
- [Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detection](https://mlsanity.com/paper/2501.16718/) (2025)
- [On Convolutions, Intrinsic Dimension, and Diffusion Models](https://mlsanity.com/paper/2506.20705/) (2025)
- [Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm](https://mlsanity.com/paper/2503.07330/) (2025)
- [A Wiener Process Perspective on Local Intrinsic Dimension Estimation Methods](https://mlsanity.com/paper/2406.17125/) (2024)
- [Safeguarding AI in Medical Imaging: Post-Hoc Out-of-Distribution Detection with Normalizing Flows](https://mlsanity.com/paper/2502.11638/) (2025)
- [Time-reversed Flow Matching with Worst Transport in High-dimensional Latent Space for Image Anomaly Detection](https://mlsanity.com/paper/2508.05461/) (2025)
- [Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation](https://mlsanity.com/paper/2605.01221/) (2026)
- [Flowing with Confidence](https://mlsanity.com/paper/2605.18472/) (2026)
- [The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spaces](https://mlsanity.com/paper/2605.22496/) (2026)
- [Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/2409.03801/) (2024)
- [Mitigating the Likelihood Paradox in Flow-based OOD Detection via Entropy Manipulation](https://mlsanity.com/paper/2602.09581/) (2026)
- [Why the Counterintuitive Phenomenon of Likelihood Rarely Appears in Tabular Anomaly Detection with Deep Generative Models?](https://mlsanity.com/paper/2602.09593/) (2026)
- [The Invisible Gorilla Effect in Out-of-distribution Detection](https://mlsanity.com/paper/2602.20068/) (2026)
- [Tippett-minimum Fusion of Representation-space Diffusion Models for Multi-Encoder Out-of-Distribution Detection](https://mlsanity.com/paper/2605.20502/) (2026)
- [Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection](https://mlsanity.com/paper/2606.00684/) (2026)
- [OOD Detection for EEG-based Machine Learning in High-Risk Environments](https://mlsanity.com/paper/2608.17620/) (2026)

## Abstract (arXiv)

Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour: when trained on a relatively complex dataset, they assign higher likelihood values to out-of-distribution (OOD) data from simpler sources. Adding to the mystery, OOD samples are never generated by these DGMs despite having higher likelihoods. This two-pronged paradox has yet to be conclusively explained, making likelihood-based OOD detection unreliable. Our primary observation is that high-likelihood regions will not be generated if they contain minimal probability mass. We demonstrate how this seeming contradiction of large densities yet low probability mass can occur around data confined to low-dimensional manifolds. We also show that this scenario can be identified through local intrinsic dimension (LID) estimation, and propose a method for OOD detection which pairs the likelihoods and LID estimates obtained from a pre-trained DGM. Our method can be applied to normalizing flows and score-based diffusion models, and obtains results which match or surpass state-of-the-art OOD detection benchmarks using the same DGM backbones. Our code is available at https://github.com/layer6ai-labs/dgm_ood_detection.