mlsanityData to 7 Oct 2026

Provably Robust Detection of Out-of-distribution Data (almost) for free

arXiv 2106.04260 ↗First posted 8 Jun 2021cs.LG cs.AI cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

The application of machine learning in safety-critical systems requires a reliable assessment of uncertainty. However, deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. Even if trained to be non-confident on OOD data, one can still adversarially manipulate OOD data so that the classifier again assigns high confidence to the manipulated samples. We show that two previously published defenses can be broken by better adapted attacks, highlighting the importance of robustness guarantees around OOD data. Since the existing method for this task is hard to train and significantly limits accuracy, we construct a classifier that can simultaneously achieve provably adversarially robust OOD detection and high clean accuracy. Moreover, by slightly modifying the classifier's architecture our method provably avoids the asymptotic overconfidence problem of standard neural networks. We provide code for all our experiments.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

42 of 61 references.

References · 42

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  10. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  12. 2020Energy-based Out-of-distribution Detectioncited by 1,296
30 more
  1. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  2. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  3. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  4. 2018Robustness May Be at Odds with Accuracycited by 873
  5. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  6. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  7. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  8. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  10. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  11. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  12. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  13. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  14. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  15. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  16. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 253
  17. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  18. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  19. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  20. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  21. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  22. 2017Maximum Resilience of Artificial Neural Networkscited by 97
  23. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  24. 2019Towards neural networks that provably know when they don't knowcited by 69
  25. 2020Enabling certification of verification-agnostic networks via memory-efficient semidefinite programmingcited by 65
  26. 2020Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?cited by 39
  27. 2021Make Sure You're Unsure: A Framework for Verifying Probabilistic Specificationscited by 18
  28. 2020Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examplescited by 16
  29. 2021Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Losscited by 11
  30. 2019Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examplescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 3; 2024: 2; 2025: 2; 2026: 2.

17 papers.

Sort:

  1. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  2. 2023In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluationcited by 132
  3. 2023Provably Bounding Neural Network Preimagescited by 28
  4. 2022Your Out-of-Distribution Detection Method is Not Robust!cited by 16
  5. 2025RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samplescited by 7
  6. 2024Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Surveycited by 6
  7. 2024Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddingscited by 6
  8. 2023Diffusion Denoised Smoothing for Certified and Adversarial Robust Out-Of-Distribution Detectioncited by 5
  9. 2021Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learningcited by 4
  10. 2022On Attacking Out-Domain Uncertainty Estimation in Deep Neural Networkscited by 4
  11. 2021FROB: Few-shot ROBust Model for Classification and Out-of-Distribution Detectioncited by 2
  12. 2022Attributing AUC-ROC to Analyze Binary Classifier Performancecited by 2
5 more
  1. 2022Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separationcited by 2
  2. 2025Open Set Label Shift with Test Time Out-of-Distribution Referencecited by 1
  3. 2026ICONIC-444: A 3.1-Million-Image Dataset for OOD Detection Researchcited by 1
  4. 2021DOODLER: Determining Out-Of-Distribution Likelihood from Encoder Reconstructionscited by 0
  5. 2026Vision-Language Model Confidence Is Not a Property of the Answercited by 0