mlsanityData to 14 Oct 2026

On Out-of-distribution Detection with Energy-based Models

arXiv 2107.08785 ↗First posted 3 Jul 2021cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

Several density estimation methods have shown to fail to detect out-of-distribution (OOD) samples by assigning higher likelihoods to anomalous data. Energy-based models (EBMs) are flexible, unnormalized density models which seem to be able to improve upon this failure mode. In this work, we provide an extensive study investigating OOD detection with EBMs trained with different approaches on tabular and image data and find that EBMs do not provide consistent advantages. We hypothesize that EBMs do not learn semantic features despite their discriminative structure similar to Normalizing Flows. To verify this hypotheses, we show that supervision and architectural restrictions improve the OOD detection of EBMs independent of the training approach.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 37 of its 56 references that are in the map. First posted on arXiv 3 Jul 2021.

Builds on

37 of 56 references.

References · 37

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2016Wide Residual Networkscited by 3,070
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  9. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  12. 2020Energy-based Out-of-distribution Detectioncited by 1,299
25 more
  1. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  2. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  3. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  4. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  5. 2018Deep Learning for Classical Japanese Literaturecited by 422
  6. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  7. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  8. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  9. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  10. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  11. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 221
  12. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  13. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 164
  14. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
  15. 2016Deep Structured Energy Based Models for Anomaly Detectioncited by 114
  16. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  17. 2019Flow Contrastive Estimation of Energy-Based Modelscited by 75
  18. 2020Density of States Estimation for Out-of-Distribution Detectioncited by 74
  19. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  20. 2020No MCMC for me: Amortized sampling for fast and stable training of energy-based modelscited by 64
  21. 2018Unbiased Implicit Variational Inferencecited by 33
  22. 2018Conditional Noise-Contrastive Estimation of Unnormalised Modelscited by 25
  23. 2016Engineering Safety in Machine Learningcited by 24
  24. 2019Prescribed Generative Adversarial Networkscited by 23
  25. 2018Building robust classifiers through generation of confident out of distribution examplescited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 6; 2025: 5; 2026: 1.

17 papers.

Sort:

By area:

  1. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  2. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50
  3. 2023Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detectioncited by 9
  4. 2024A Unified Approach Towards Active Learning and Out-of-Distribution Detectioncited by 7
  5. 2025Joint Out-of-Distribution Filtering and Data Discovery Active Learningcited by 7
  6. 2023Persistently Trained, Diffusion-assisted Energy-based Modelscited by 4
  7. 2024Latent Space Energy-based Neural ODEscited by 4
  8. 2025An Information-Theoretic Framework for Feature Construction in Out-of-Distribution Detectioncited by 4
  9. 2025BOOD: Boundary-based Out-Of-Distribution Data Generationcited by 4
  10. 2024Federated Learning with Heterogeneous Data Handling for Robust Vehicular Object Detectioncited by 2
  11. 2026Tuning Out-of-Distribution (OOD) Detectors Without Given OOD Datacited by 2
  12. 2022Mitigating Out-of-Distribution Data Density Overestimation in Energy-Based Modelscited by 1
5 more
  1. 2022EBMs vs. CL: Exploring Self-Supervised Visual Pretraining for Visual Question Answeringcited by 0
  2. 2024Exploring Energy-Based Models for Out-of-Distribution Detection in Dialect Identificationcited by 0
  3. 2024DNN-GDITD: Out-of-distribution detection via Deep Neural Network based Gaussian Descriptor for Imbalanced Tabular Datacited by 0
  4. 2025Score-Based Training for Energy-Based TTS Modelscited by 0
  5. 2025ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimationcited by 0