mlsanityData to 7 Oct 2026

Out-of-Distribution Detection with Reconstruction Error and Typicality-based Penalty

arXiv 2212.12641 ↗First posted 24 Dec 2022cs.LG cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

The task of out-of-distribution (OOD) detection is vital to realize safe and reliable operation for real-world applications. After the failure of likelihood-based detection in high dimensions had been shown, approaches based on the typical set have been attracting attention; however, they still have not achieved satisfactory performance. Beginning by presenting the failure case of the typicality-based approach, we propose a new reconstruction error-based approach that employs normalizing flow (NF). We further introduce a typicality-based penalty, and by incorporating it into the reconstruction error in NF, we propose a new OOD detection method, penalized reconstruction error (PRE). Because the PRE detects test inputs that lie off the in-distribution manifold, it effectively detects adversarial examples as well as OOD examples. We show the effectiveness of our method through the evaluation using natural image datasets, CIFAR-10, TinyImageNet, and ILSVRC2012.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 59 of its 86 references that are in the map. First posted on arXiv 24 Dec 2022.

Builds on

59 of 86 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2016Wide Residual Networkscited by 3,057
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  11. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  12. 2015Variational Inference with Normalizing Flowscited by 2,161
38 more
  1. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  2. 2016Density estimation using Real NVPcited by 2,029
  3. 2016Adversarial examples in the physical worldcited by 1,762
  4. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  10. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  11. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  12. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  13. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  14. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  15. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  16. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
  17. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  18. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  19. 2018Assessing Generative Models via Precision and Recallcited by 401
  20. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  21. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
  22. 2017Detecting Adversarial Samples from Artifactscited by 244
  23. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 243
  24. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 215
  25. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  26. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  27. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  28. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  29. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  30. 2019Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancycited by 112
  31. 2018Generative Probabilistic Novelty Detection with Adversarial Autoencoderscited by 99
  32. 2021Understanding Failures in Out-of-Distribution Detection with Deep Generative Modelscited by 89
  33. 2018IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesiscited by 85
  34. 2019The Odds are Odd: A Statistical Test for Detecting Adversarial Examplescited by 75
  35. 2020Density of States Estimation for Out-of-Distribution Detectioncited by 74
  36. 2020Understanding and Mitigating Exploding Inverses in Invertible Neural Networkscited by 71
  37. 2016A Boundary Tilting Persepective on the Phenomenon of Adversarial Examplescited by 70
  38. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 7; 2025: 2.

11 papers.

  1. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  2. 2024What makes an image realistic?cited by 10
  3. 2023Unsupervised Recognition of Unknown Objects for Open-World Object Detectioncited by 6
  4. 2024Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Surveycited by 6
  5. 2024Diffusion-based Layer-wise Semantic Reconstruction for Unsupervised Out-of-Distribution Detectioncited by 5
  6. 2024Persistent Classification: A New Approach to Stability of Data and Adversarial Examplescited by 3
  7. 2025InvAD: Inversion-based Reconstruction-Free Anomaly Detection with Diffusion Modelscited by 3
  8. 2023SR-OOD: Out-of-Distribution Detection via Sample Repairingcited by 1
  9. 2024Learning Non-Linear Invariants for Unsupervised Out-of-Distribution Detectioncited by 0
  10. 2024Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detectioncited by 0
  11. 2025Out of Distribution Detection for Efficient Continual Learning in Quality Prediction for Arc Weldingcited by 0