mlsanityData to 7 Oct 2026

A Data-Driven Measure of Relative Uncertainty for Misclassification Detection

arXiv 2306.01710 ↗First posted 2 Jun 2023stat.ML cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Misclassification detection is an important problem in machine learning, as it allows for the identification of instances where the model's predictions are unreliable. However, conventional uncertainty measures such as Shannon entropy do not provide an effective way to infer the real uncertainty associated with the model's predictions. In this paper, we introduce a novel data-driven measure of uncertainty relative to an observer for misclassification detection. By learning patterns in the distribution of soft-predictions, our uncertainty measure can identify misclassified samples based on the predicted class probabilities. Interestingly, according to the proposed measure, soft-predictions corresponding to misclassified instances can carry a large amount of uncertainty, even though they may have low Shannon entropy. We demonstrate empirical improvements over multiple image classification tasks, outperforming state-of-the-art misclassification detection methods.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

39 of 51 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2021Convex optimizationcited by 2,479
  8. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  9. 2016Concrete Problems in AI Safetycited by 1,582
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  12. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
27 more
  1. 2017Selective Classification for Deep Neural Networkscited by 799
  2. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  3. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 459
  4. 2020Classification with Valid and Adaptive Coveragecited by 390
  5. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  6. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  7. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  8. 2019Deep Learning for Anomaly Detection: A Surveycited by 357
  9. 2018Recent Advances in Open Set Recognition: A Surveycited by 309
  10. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  11. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  12. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 206
  13. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  14. 2019Addressing Failure Prediction by Learning Model Confidencecited by 183
  15. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 62
  16. 2021DOCTOR: A Simple Method for Detecting Misclassification Errorscited by 52
  17. 2023Rethinking Confidence Calibration for Failure Predictioncited by 43
  18. 2021Neural Mean Discrepancy for Efficient Out-of-Distribution Detectioncited by 42
  19. 2019Probabilistic Modeling of Deep Features for Out-of-Distribution and Adversarial Detectioncited by 39
  20. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  21. 2022Nonparametric Uncertainty Quantification for Single Deterministic Neural Networkcited by 36
  22. 2022RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustnesscited by 27
  23. 2021Deep Deterministic Uncertainty for Semantic Segmentationcited by 24
  24. 2022Single Model Uncertainty Estimation via Stochastic Data Centeringcited by 20
  25. 2023The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognitioncited by 16
  26. 2020URSABench: Comprehensive Benchmarking of Approximate Bayesian Inference Methods for Deep Neural Networkscited by 9
  27. 2022Context-Aware Drift Detectioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 4; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  2. 2025ViLU: Learning Vision-Language Uncertainties for Failure Predictioncited by 9
  3. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  4. 2023SwiftLearn: A Data-Efficient Training Method of Deep Learning Models using Importance Samplingcited by 3
  5. 2024BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptationcited by 3
  6. 2024Beyond the Norms: Detecting Prediction Errors in Regression Modelscited by 2
  7. 2025Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassificationscited by 2
  8. 2024Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEEcited by 1
  9. 2026Is the Last Layer Sufficient for Uncertainty Quantification?cited by 1
  10. 2025RAT: Boosting Misclassification Detection Ability without Extra Datacited by 0
  11. 2026X-MAP: eXplainable Misclassification Analysis and Profiling for Spam and Phishing Detectioncited by 0
  12. 2026LogitDynamics: Reliable ViT Error Detection from Layerwise Logit Trajectoriescited by 0
2 more
  1. 2026A Convolutional Layer Activation Dimensionality Reduction for Out-of-Distribution and Adversarial Attack Detection Methodscited by 0
  2. 2026FailSAE: Towards Interpretable Failure Prediction for Vision-Language Models via Sparse Autoencoderscited by 0