mlsanityData to 14 Oct 2026

A Geometric Explanation of the Likelihood OOD Detection Paradox

arXiv 2403.18910 ↗First posted 27 Mar 2024cs.LG cs.AI cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour: when trained on a relatively complex dataset, they assign higher likelihood values to out-of-distribution (OOD) data from simpler sources. Adding to the mystery, OOD samples are never generated by these DGMs despite having higher likelihoods. This two-pronged paradox has yet to be conclusively explained, making likelihood-based OOD detection unreliable. Our primary observation is that high-likelihood regions will not be generated if they contain minimal probability mass. We demonstrate how this seeming contradiction of large densities yet low probability mass can occur around data confined to low-dimensional manifolds. We also show that this scenario can be identified through local intrinsic dimension (LID) estimation, and propose a method for OOD detection which pairs the likelihoods and LID estimates obtained from a pre-trained DGM. Our method can be applied to normalizing flows and score-based diffusion models, and obtains results which match or surpass state-of-the-art OOD detection benchmarks using the same DGM backbones. Our code is available at https://github.com/layer6ai-labs/dgm_ood_detection.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 57 of its 87 references that are in the map. First posted on arXiv 27 Mar 2024.

Builds on

57 of 87 references.

References · 57

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2018Neural Ordinary Differential Equationscited by 3,779
  9. 2016Density estimation using Real NVPcited by 2,042
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  12. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,186
38 more
  1. 2016End to End Learning for Self-Driving Carscited by 1,009
  2. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  3. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  4. 2021Score-based Generative Modeling in Latent Spacecited by 636
  5. 2019Neural Spline Flowscited by 596
  6. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  7. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  8. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  9. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 315
  10. 2019Deep Semi-Supervised Anomaly Detectioncited by 291
  11. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 252
  12. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 250
  13. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  14. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 221
  15. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  16. 2016A Theory of Generative ConvNetcited by 174
  17. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  18. 2022Score-Based Generative Models Detect Manifoldscited by 122
  19. 2020Flows for simultaneous manifold learning and density estimationcited by 118
  20. 2022Denoising diffusion models for out-of-distribution detectioncited by 104
  21. 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
  22. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  23. 2020Density of States Estimation for Out-of-Distribution Detectioncited by 74
  24. 2014Toward Supervised Anomaly Detectioncited by 73
  25. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
  26. 2021Scikit-dimension: a Python package for intrinsic dimension estimationcited by 68
  27. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  28. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 64
  29. 2018Universal features of price formation in financial markets: perspectives from Deep Learningcited by 60
  30. 2023Unsupervised Out-of-Distribution Detection with Diffusion Inpaintingcited by 59
  31. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  32. 2021Rectangular Flows for Manifold Learningcited by 50
  33. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  34. 2019Neural Empirical Bayescited by 46
  35. 2020Perfect density models cannot guarantee anomaly detectioncited by 39
  36. 2022Your diffusion model secretly knows the dimension of the data manifoldcited by 36
  37. 2021Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flowscited by 31
  38. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 5; 2026: 9.

22 papers.

Sort:

By area:

  1. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  2. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 42
  3. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  4. 2024Losing dimensions: Geometric memorization in generative diffusioncited by 29
  5. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26
  6. 2024What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Modelscited by 7
  7. 2025Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detectioncited by 4
  8. 2025On Convolutions, Intrinsic Dimension, and Diffusion Modelscited by 3
  9. 2025Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigmcited by 2
  10. 2024A Wiener Process Perspective on Local Intrinsic Dimension Estimation Methodscited by 1
  11. 2025Safeguarding AI in Medical Imaging: Post-Hoc Out-of-Distribution Detection with Normalizing Flowscited by 1
  12. 2025Time-reversed Flow Matching with Worst Transport in High-dimensional Latent Space for Image Anomaly Detectioncited by 1
10 more
  1. 2026Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimationcited by 1
  2. 2026Flowing with Confidencecited by 1
  3. 2026The Signal in the Noise: OOD Detection Through Goodness-of-Fit Testing in Factorised Latent Spacescited by 1
  4. 2024Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detectioncited by 0
  5. 2026Mitigating the Likelihood Paradox in Flow-based OOD Detection via Entropy Manipulationcited by 0
  6. 2026Why the Counterintuitive Phenomenon of Likelihood Rarely Appears in Tabular Anomaly Detection with Deep Generative Models?cited by 0
  7. 2026The Invisible Gorilla Effect in Out-of-distribution Detectioncited by 0
  8. 2026Tippett-minimum Fusion of Representation-space Diffusion Models for Multi-Encoder Out-of-Distribution Detectioncited by 0
  9. 2026Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detectioncited by 0
  10. 2026OOD Detection for EEG-based Machine Learning in High-Risk Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting Likelihood-Based Out-of-Distribution Detection by Modeling Representations
  2. 2025Probability Density from Latent Diffusion Models for Out-of-Distribution Detection
  3. 2024Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection
  4. 2026Geometry over Density: Few-Shot Cross-Domain OOD Detection
  5. 2026Deep Networks Favor Simple Data
  6. 2024Beyond Perceptual Distance: Discrepancy Assessment on Deep Representation for Out-of-Distribution Detection with Diffusion Model
  7. 2024Non-Linear Outlier Synthesis for Out-of-Distribution Detection
  8. 2024Can OOD Object Detectors Learn from Foundation Models?