mlsanityData to 14 Oct 2026

Promises and Pitfalls of Black-Box Concept Learning Models

arXiv 2106.13314 ↗First posted 24 Jun 2021cs.LG cs.AIcited by 131

Not on the map. Why?

Abstract (arXiv)

Machine learning models that incorporate concept learning as an intermediate step in their decision making process can match the performance of black-box predictive models while retaining the ability to explain outcomes in human understandable terms. However, we demonstrate that the concept representations learned by these models encode information beyond the pre-defined concepts, and that natural mitigation strategies do not fully work, rendering the interpretation of the downstream prediction misleading. We describe the mechanism underlying the information leakage and suggest recourse for mitigating its effects.

Key facts. Cited by 131 papers in the map (data through 14 Oct 2026). Builds on 8 of its 12 references that are in the map. First posted on arXiv 24 Jun 2021.

Builds on

8 of 12 references.

References · 8

  1. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  2. 2020Concept Bottleneck Modelscited by 1,032
  3. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  4. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  5. 2021Do Concept Bottleneck Models Learn as Intended?cited by 116
  6. 2018Learning Latent Subspaces in Variational Autoencoderscited by 51
  7. 2019Interpretability Beyond Classification Output: Semantic Bottleneck Networkscited by 49
  8. 2018Semantic bottleneck for computer vision taskscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 14; 2023: 20; 2024: 29; 2025: 32; 2026: 34.

131 papers.

Sort:

By area:

  1. 2022Post-hoc Concept Bottleneck Modelscited by 293
  2. 2023Getting aligned on representational alignmentcited by 143
  3. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  4. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  5. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
  6. 2023Probabilistic Concept Bottleneck Modelscited by 101
  7. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
  8. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  9. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  10. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  11. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  12. 2022Concept Embedding Models: Beyond the Accuracy-Explainability Trade-Offcited by 61
38 more
  1. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  2. 2023Interpretable Neural-Symbolic Concept Reasoningcited by 57
  3. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  4. 2024Stochastic Concept Bottleneck Modelscited by 51
  5. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  6. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  7. 2023Human Uncertainty in Concept-Based AI Systemscited by 39
  8. 2023Auxiliary Losses for Learning Generalizable Concept-based Modelscited by 35
  9. 2023Towards Robust Metrics for Concept Representation Evaluationcited by 29
  10. 2023Learning to Intervene on Concept Bottleneckscited by 29
  11. 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 29
  12. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  13. 2023SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel Histopathologycited by 25
  14. 2022Human-Centered Concept Explanations for Neural Networkscited by 23
  15. 2022Understanding and Enhancing Robustness of Concept-based Modelscited by 22
  16. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  17. 2023Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneckcited by 22
  18. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 21
  19. 2022Concept Embedding Analysis: A Reviewcited by 20
  20. 2024A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcutscited by 19
  21. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 18
  22. 2024Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learningcited by 17
  23. 2024Concept Bottleneck Models Without Predefined Conceptscited by 17
  24. 2024Bayesian Concept Bottleneck Models with LLM Priorscited by 17
  25. 2023Relational Concept Bottleneck Modelscited by 15
  26. 2024Concept Bottleneck Language Models For protein designcited by 15
  27. 2025If Concept Bottlenecks are the Question, are Foundation Models the Answer?cited by 15
  28. 2024Neural Concept Bindercited by 14
  29. 2024Understanding Inter-Concept Relationships in Concept-Based Modelscited by 13
  30. 2025Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lenscited by 13
  31. 2025Leakage and Interpretability in Concept-Based Modelscited by 13
  32. 2022Adapting to Latent Subgroup Shifts via Concepts and Proxiescited by 12
  33. 2023Cross-Modal Conceptualization in Bottleneck Modelscited by 12
  34. 2023Interpretable and intervenable ultrasonography-based machine learning models for pediatric appendicitiscited by 11
  35. 2024Do Concept Bottleneck Models Respect Localities?cited by 10
  36. 2025Measuring Leakage in Concept-Based Methods: An Information Theoretic Approachcited by 9
  37. 2025Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shiftscited by 9
  38. 2024A survey on Concept-based Approaches For Model Improvementcited by 8

50 most cited of 131 All 131 (JSON)