mlsanityData to 7 Oct 2026

DOCTOR: A Simple Method for Detecting Misclassification Errors

arXiv 2106.02395 ↗First posted 4 Jun 2021cs.CV cs.LGcited by 52

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have shown to perform very well on large scale object recognition problems and lead to widespread use for real-world applications, including situations where DNN are implemented as "black boxes". A promising approach to secure their use is to accept decisions that are likely to be correct while discarding the others. In this work, we propose DOCTOR, a simple method that aims to identify whether the prediction of a DNN classifier should (or should not) be trusted so that, consequently, it would be possible to accept it or to reject it. Two scenarios are investigated: Totally Black Box (TBB) where only the soft-predictions are available and Partially Black Box (PBB) where gradient-propagation to perform input pre-processing is allowed. Empirically, we show that DOCTOR outperforms all state-of-the-art methods on various well-known images and sentiment analysis datasets. In particular, we observe a reduction of up to 4% of the false rejection rate (FRR) in the PBB scenario. DOCTOR can be applied to any pre-trained model, it does not require prior information about the underlying dataset and is as simple as the simplest available methods in the literature.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

23 of 36 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  10. 2017Selective Classification for Deep Neural Networkscited by 799
  11. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  12. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
11 more
  1. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  2. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  3. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  4. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  5. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  6. 2018Bias-Reduced Uncertainty Estimation for Deep Neural Classifierscited by 106
  7. 2020Robust Out-of-distribution Detection for Neural Networkscited by 57
  8. 2020Selective Classification via One-Sided Predictioncited by 37
  9. 2019Distance-Based Learning from Errors for Confidence Calibrationcited by 23
  10. 2019Confidence Calibration for Convolutional Neural Networks Using Structured Dropoutcited by 19
  11. 2016Estimating Uncertainty Online Against an Adversarycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 11; 2024: 12; 2025: 9; 2026: 11.

52 papers.

Sort:

By area:

  1. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  2. 2022Augmenting Softmax Information for Selective Classification with Out-of-Distribution Datacited by 27
  3. 2022Out-Of-Distribution Detection Is Not All You Needcited by 23
  4. 2022A Note on "Assessing Generalization of SGD via Disagreement"cited by 16
  5. 2023The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognitioncited by 16
  6. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 15
  7. 2023How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networkscited by 15
  8. 2023Plugin estimators for selective classification with out-of-distribution detectioncited by 14
  9. 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
  10. 2022Failure Detection in Medical Image Classification: A Reality Check and Benchmarking Testbedcited by 12
  11. 2024Selective Classification Under Distribution Shiftscited by 12
  12. 2024Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix Itcited by 11
38 more
  1. 2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
  2. 2024Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifierscited by 10
  3. 2025ViLU: Learning Vision-Language Uncertainties for Failure Predictioncited by 9
  4. 2022Unifying Evaluation of Machine Learning Safety Monitorscited by 8
  5. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  6. 2025Interpretable Failure Detection with Human-Level Conceptscited by 7
  7. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  8. 2022Improving Disease Classification Performance and Explainability of Deep Learning Models in Radiology with Heatmap Generatorscited by 3
  9. 2023ASPEST: Bridging the Gap Between Active Learning and Selective Predictioncited by 3
  10. 2023Expecting The Unexpected: Towards Broad Out-Of-Distribution Detectioncited by 3
  11. 2024SCOD: From Heuristics to Theorycited by 3
  12. 2024Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errorscited by 3
  13. 2024Safety Monitoring of Machine Learning Perception Functions: a Surveycited by 3
  14. 2025To Trust Or Not To Trust Your Vision-Language Model's Predictioncited by 3
  15. 2022MEAD: A Multi-Armed Approach for Evaluation of Adversarial Examples Detectorscited by 2
  16. 2023Reject option models comprising out-of-distribution detectioncited by 2
  17. 2024Beyond the Norms: Detecting Prediction Errors in Regression Modelscited by 2
  18. 2025Model-agnostic Selective Labeling with Provable Statistical Guaranteescited by 2
  19. 2025Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassificationscited by 2
  20. 2026Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answeringcited by 2
  21. 2024Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEEcited by 1
  22. 2024Combine and Conquer: A Meta-Analysis on Data Shift and Out-of-Distribution Detectioncited by 1
  23. 2026Adaptive Confidence Regularization for Multimodal Failure Detectioncited by 1
  24. 2026Is the Last Layer Sufficient for Uncertainty Quantification?cited by 1
  25. 2022Constraining Representations Yields Models That Know What They Don't Knowcited by 0
  26. 2022Incorporating Crowdsourced Annotator Distributions into Ensemble Modeling to Improve Classification Trustworthiness for Ancient Greek Papyricited by 0
  27. 2023A Minimax Approach Against Multi-Armed Adversarial Attacks Detectioncited by 0
  28. 2023Be Bayesian by Attachments to Catch More Uncertaintycited by 0
  29. 2024Optimal Zero-Shot Detector for Multi-Armed Attackscited by 0
  30. 2025RAT: Boosting Misclassification Detection Ability without Extra Datacited by 0
  31. 2025EVINET: Towards Open-World Graph Learning via Evidential Reasoning Networkcited by 0
  32. 2025A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shiftscited by 0
  33. 2026Selective Prior Synchronization via SYNC Losscited by 0
  34. 2026X-MAP: eXplainable Misclassification Analysis and Profiling for Spam and Phishing Detectioncited by 0
  35. 2026Trustworthy Endoscopic Super-Resolutioncited by 0
  36. 2026Post-hoc Selective Classification for Reliable Synthetic Image Detectioncited by 0
  37. 2026Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learningcited by 0
  38. 2026Beyond Uncertainty: Generalizable Failure Monitoring for Surgical Segmentation under Acquisition Degradationcited by 0

50 most cited of 52 All 52 (JSON)