mlsanityData to 7 Oct 2026

Concept-based Explanations for Out-Of-Distribution Detectors

arXiv 2203.02586 ↗First posted 4 Mar 2022cs.LG cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection plays a crucial role in ensuring the safe deployment of deep neural network (DNN) classifiers. While a myriad of methods have focused on improving the performance of OOD detectors, a critical gap remains in interpreting their decisions. We help bridge this gap by providing explanations for OOD detectors based on learned high-level concepts. We first propose two new metrics for assessing the effectiveness of a particular set of concepts for explaining OOD detectors: 1) detection completeness, which quantifies the sufficiency of concepts for explaining an OOD-detector's decisions, and 2) concept separability, which captures the distributional separation between in-distribution and OOD data in the concept space. Based on these metrics, we propose an unsupervised framework for learning a set of concepts that satisfy the desired properties of high detection completeness and concept separability, and demonstrate its effectiveness in providing concept-based explanations for diverse off-the-shelf OOD detectors. We also show how to identify prominent concepts contributing to the detection results, and provide further reasoning about their decisions.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 51 of its 76 references that are in the map. First posted on arXiv 4 Mar 2022.

Builds on

51 of 76 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2013Describing Textures in the Wildcited by 2,110
  12. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
38 more
  1. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  2. 2016Concrete Problems in AI Safetycited by 1,582
  3. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  7. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  8. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  9. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  10. 2020Concept Bottleneck Modelscited by 1,022
  11. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  12. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  13. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  14. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  15. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  16. 2013Do Deep Nets Really Need to be Deep?cited by 466
  17. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  18. 2017Interpretation of Neural Networks is Fragilecited by 443
  19. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  20. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  21. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  22. 2019The many Shapley values for model explanationcited by 341
  23. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  24. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  25. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  26. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  27. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  28. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  29. 2020Debugging Tests for Model Explanationscited by 146
  30. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  31. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  32. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  33. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  34. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  35. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  36. 2015Posterior calibration and exploratory analysis for natural language processing modelscited by 75
  37. 2017Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Netcited by 27
  38. 2020A General Framework For Detecting Anomalous Inputs to DNN Classifierscited by 24

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 5; 2025: 3; 2026: 3.

16 papers.

Sort:

  1. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
  2. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 21
  3. 2024Do Concept Bottleneck Models Respect Localities?cited by 10
  4. 2024Adaptive Concept Bottleneck for Foundation Models Under Distribution Shiftscited by 9
  5. 2025Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shiftscited by 9
  6. 2022Assessing Out-of-Domain Language Model Performance from Few Examplescited by 6
  7. 2024Incorporating Expert Rules into Neural Networks in the Framework of Concept-Based Learningcited by 3
  8. 2024The AI-DEC: A Card-based Design Method for User-centered AI Explanationscited by 2
  9. 2023Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shiftscited by 1
  10. 2023Estimation of Concept Explanations Should be Uncertainty Awarecited by 1
  11. 2024FI-CBL: A Probabilistic Method for Concept-Based Learning with Expert Rulescited by 1
  12. 2025ConceptFlow: Hierarchical and Fine-grained Concept-Based Explanation for Convolutional Neural Networkscited by 0
4 more
  1. 2025Explainable Visual Anomaly Detection via Concept Bottleneck Modelscited by 0
  2. 2026SeNeDiF-OOD: Semantic Nested Dichotomy Fusion for Out-of-Distribution Detection Methodology in Open-World Classification. A Case Study on Monument Style Classificationcited by 0
  3. 2026When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbationscited by 0
  4. 2026Sparse Autoencoders for Interpretable Out-of-Distribution Detectioncited by 0