mlsanityData to 14 Oct 2026

Toward a Unified Framework for Debugging Concept-based Models

arXiv 2109.11160 ↗First posted 23 Sep 2021cs.LG cs.SEcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we tackle interactive debugging of "gray-box" concept-based models (CBMs). These models learn task-relevant concepts appearing in the inputs and then compute a prediction by aggregating the concept activations. Our work stems from the observation that in CBMs both the concepts and the aggregation function can be affected by different kinds of bugs, and that fixing these bugs requires different kinds of corrective supervision. To this end, we introduce a simple schema for human supervisors to identify and prioritize bugs in both components, and discuss solution strategies and open problems. We also introduce a novel loss function for debugging the aggregation step that generalizes existing strategies for aligning black-box models to CBMs by making them robust to how the concepts change during training.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

32 of 50 references.

References · 32

  1. 2013Intriguing properties of neural networkscited by 5,155
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2020Knowledge Distillation: A Surveycited by 1,603
  5. 2020Concept Bottleneck Modelscited by 1,032
  6. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  7. 2016The Mythos of Model Interpretabilitycited by 721
  8. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  9. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  10. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  11. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  12. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
20 more
  1. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  2. 2019Explanations can be manipulated and geometry is to blamecited by 216
  3. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  4. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  5. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
  6. 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
  7. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  8. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  9. 2020On the Tractability of SHAP Explanationscited by 92
  10. 2019Interpretable Image Recognition with Hierarchical Prototypescited by 65
  11. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  12. 2020This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognitioncited by 50
  13. 2019Interpretability Beyond Classification Output: Semantic Bottleneck Networkscited by 49
  14. 2021Interactive Label Cleaning with Example-based Explanationscited by 49
  15. 2020FIND: Human-in-the-Loop Debugging Deep Text Classifierscited by 40
  16. 2020Debiasing Concept-based Explanations with Causal Analysiscited by 32
  17. 2017Contextual Explanation Networkscited by 21
  18. 2021Dissonance Between Human and Machine Understandingcited by 18
  19. 2020Learning Interpretable Concept-Based Models with Human Feedbackcited by 17
  20. 2020Machine Guides, Human Supervises: Interactive Learning with Global Explanationscited by 16

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 1.

4 papers.

  1. 2023Learning to Intervene on Concept Bottleneckscited by 29
  2. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  3. 2025Debugging Concept Bottleneck Models through Removal and Retrainingcited by 2
  4. 2022AI Model Utilization Measurements For Finding Class Encoding Patternscited by 0