mlsanityData to 14 Oct 2026

Rethinking Test-time Likelihood: The Likelihood Path Principle and Its Application to OOD Detection

arXiv 2401.04933 ↗First posted 10 Jan 2024cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

While likelihood is attractive in theory, its estimates by deep generative models (DGMs) are often broken in practice, and perform poorly for out of distribution (OOD) Detection. Various recent works started to consider alternative scores and achieved better performances. However, such recipes do not come with provable guarantees, nor is it clear that their choices extract sufficient information. We attempt to change this by conducting a case study on variational autoencoders (VAEs). First, we introduce the likelihood path (LPath) principle, generalizing the likelihood principle. This narrows the search for informative summary statistics down to the minimal sufficient statistics of VAEs' conditional likelihoods. Second, introducing new theoretic tools such as nearly essential support, essential distance and co-Lipschitzness, we obtain non-asymptotic provable OOD detection guarantees for certain distillation of the minimal sufficient statistics. The corresponding LPath algorithm demonstrates SOTA performances, even using simple and small VAEs with poor likelihood estimates. To our best knowledge, this is the first provable unsupervised OOD method that delivers excellent empirical results, better than any other VAEs based techniques. We use the same model as citexiao2020likelihood, open sourced from: https://github.com/XavierXiao/Likelihood-Regret

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

34 of 50 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  10. 2014Convex Optimization: Algorithms and Complexitycited by 575
  11. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  12. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
22 more
  1. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  2. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 221
  3. 2018Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learningcited by 207
  4. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  5. 2019Practical Deep Learning with Bayesian Principlescited by 158
  6. 2020COPOD: Copula-Based Outlier Detectioncited by 146
  7. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  8. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  9. 2022Denoising diffusion models for out-of-distribution detectioncited by 104
  10. 2015Optimal whitening and decorrelationcited by 97
  11. 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
  12. 2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
  13. 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
  14. 2020Density of States Estimation for Out-of-Distribution Detectioncited by 74
  15. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
  16. 2023Unsupervised Out-of-Distribution Detection with Diffusion Inpaintingcited by 59
  17. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  18. 2020Perfect density models cannot guarantee anomaly detectioncited by 39
  19. 2022Model-agnostic out-of-distribution detection using combined statistical testscited by 14
  20. 2020BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertaintycited by 7
  21. 2014Comment on "Detecting Novel Associations In Large Data Sets" by Reshef Et Al, Science Dec 16, 2011cited by 1
  22. 2021Label Smoothed Embedding Hypothesis for Out-of-Distribution Detectioncited by 1

Built on by

Not cited yet.