mlsanityData to 7 Oct 2026

Understanding Measures of Uncertainty for Adversarial Example Detection

arXiv 1803.08533 ↗First posted 22 Mar 2018stat.ML cs.LGcited by 177

Not on the map. Why?

Abstract (arXiv)

Measuring uncertainty is a promising technique for detecting adversarial examples, crafted inputs on which the model predicts an incorrect class with high confidence. But many measures of uncertainty exist, including predictive en- tropy and mutual information, each capturing different types of uncertainty. We study these measures, and shed light on why mutual information seems to be effective at the task of adversarial example detection. We highlight failure modes for MC dropout, a widely used approach for estimating uncertainty in deep models. This leads to an improved understanding of the drawbacks of current methods, and a proposal to improve the quality of uncertainty estimates using probabilistic model ensembles. We give illustrative experiments using MNIST to demonstrate the intuition underlying the different measures of uncertainty, as well as experiments on a real world Kaggle dogs vs cats classification dataset.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 32; 2023: 38; 2024: 32; 2025: 25; 2026: 18.

177 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  3. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  4. 2022Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?cited by 124
  5. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  6. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  7. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  8. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  9. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  10. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
  11. 2021Improving black-box optimization in VAE latent space using decoder uncertaintycited by 51
  12. 2022A Call to Reflect on Evaluation Practices for Failure Detection in Image Classificationcited by 51
38 more
  1. 2022Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Taskscited by 49
  2. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  3. 2021Posterior Meta-Replay for Continual Learningcited by 45
  4. 2023Credal Bayesian Deep Learningcited by 45
  5. 2021Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertaintiescited by 41
  6. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  7. 2023Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?cited by 40
  8. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 40
  9. 2022Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overviewcited by 37
  10. 2023Explaining Predictive Uncertainty with Information Theoretic Shapley Valuescited by 36
  11. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  12. 2023Second-Order Uncertainty Quantification: A Distance-Based Approachcited by 35
  13. 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
  14. 2023Learning to Reject with a Fixed Predictor: Application to Decontextualizationcited by 32
  15. 2021Detecting Adversarial Examples from Sensitivity Inconsistency of Spatial-Transform Domaincited by 29
  16. 2021QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Resultscited by 28
  17. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  18. 2023Introducing an Improved Information-Theoretic Measure of Predictive Uncertaintycited by 28
  19. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 24
  20. 2021Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outlierscited by 22
  21. 2024Label-wise Aleatoric and Epistemic Uncertainty Quantificationcited by 22
  22. 2024Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rulescited by 21
  23. 2024Uncertainty in Graph Neural Networks: A Surveycited by 20
  24. 2021Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targetscited by 19
  25. 2023Second-Order Uncertainty Quantification: Variance-Based Measurescited by 19
  26. 2021Learning Multimodal VAEs through Mutual Supervisioncited by 18
  27. 2021Dense outlier detection and open-set recognition based on training with noisy negative imagescited by 17
  28. 2022A Note on "Assessing Generalization of SGD via Disagreement"cited by 16
  29. 2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
  30. 2022Out of Distribution Data Detection Using Dropout Bayesian Neural Networkscited by 15
  31. 2023Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysiscited by 15
  32. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  33. 2021Two Coupled Rejection Metrics Can Tell Adversarial Examples Apartcited by 14
  34. 2023Evidential Uncertainty Quantification: A Variance-Based Perspectivecited by 14
  35. 2023A Novel Bayes' Theorem for Upper Probabilitiescited by 13
  36. 2022On the Robustness of Bayesian Neural Networks to Adversarial Attackscited by 12
  37. 2021Leveraging Uncertainty for Improved Static Malware Detection Under Extreme False Positive Constraintscited by 11
  38. 2023Towards Accelerated Model Training via Bayesian Data Selectioncited by 11

50 most cited of 177 All 177 (JSON)