mlsanityData to 14 Oct 2026

If Concept Bottlenecks are the Question, are Foundation Models the Answer?

arXiv 2504.19774 ↗First posted 28 Apr 2025cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Concept Bottleneck Models (CBMs) are neural networks designed to conjoin high performance with ante-hoc interpretability. CBMs work by first mapping inputs (e.g., images) to high-level concepts (e.g., visible objects and their properties) and then use these to solve a downstream task (e.g., tagging or scoring an image) in an interpretable manner. Their performance and interpretability, however, hinge on the quality of the concepts they learn. The go-to strategy for ensuring good quality concepts is to leverage expert annotations, which are expensive to collect and seldom available in applications. Researchers have recently addressed this issue by introducing "VLM-CBM" architectures that replace manual annotations with weak supervision from foundation models. It is however unclear what is the impact of doing so on the quality of the learned concepts. To answer this question, we put state-of-the-art VLM-CBMs to the test, analyzing their learned concepts empirically using a selection of significant metrics. Our results show that, depending on the task, VLM supervision can sensibly differ from expert annotations, and that concept accuracy and quality are not strongly correlated. Our code is available at https://github.com/debryu/CQA.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 65 of its 86 references that are in the map. First posted on arXiv 28 Apr 2025.

Builds on

65 of 86 references.

References · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  7. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  8. 2018Disentangling by Factorisingcited by 616
  9. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  10. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  11. 2018Towards a Definition of Disentangled Representationscited by 296
  12. 2022Post-hoc Concept Bottleneck Modelscited by 293
38 more
  1. 2023Label-Free Concept Bottleneck Modelscited by 285
  2. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  3. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  4. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  5. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 131
  6. 2024To Believe or Not to Believe Your LLMcited by 120
  7. 2021Do Concept Bottleneck Models Learn as Intended?cited by 116
  8. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  9. 2023Probabilistic Concept Bottleneck Modelscited by 101
  10. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
  11. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  12. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  13. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  14. 2022Concept Bottleneck Model with Additional Unsupervised Conceptscited by 79
  15. 2022Interactive Concept Bottleneck Modelscited by 79
  16. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  17. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  18. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  19. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  20. 2023Interpretable Neural-Symbolic Concept Reasoningcited by 57
  21. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 53
  22. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  23. 2021Entropy-based Logic Explanations of Neural Networkscited by 51
  24. 2024Stochastic Concept Bottleneck Modelscited by 51
  25. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  26. 2020FIND: Human-in-the-Loop Debugging Deep Text Classifierscited by 40
  27. 2020Debiasing Concept-based Explanations with Causal Analysiscited by 32
  28. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  29. 2023Towards Robust Metrics for Concept Representation Evaluationcited by 29
  30. 2023Learning to Intervene on Concept Bottleneckscited by 29
  31. 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learningcited by 29
  32. 2024Beyond Concept Bottleneck Models: How to Make Black Boxes Intervenable?cited by 28
  33. 2024Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Modelscited by 25
  34. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  35. 2022Concept Embedding Analysis: A Reviewcited by 20
  36. 2022Unpacking Large Language Models with Conceptual Consistencycited by 18
  37. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  38. 2021Is Disentanglement all you need? Comparing Concept-based & Disentanglement Approachescited by 17

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Object Centric Concept Bottleneckscited by 10
  2. 2025There Was Never a Bottleneck in Concept Bottleneck Modelscited by 8
  3. 2025Enhancing Concept Localization in CLIP-based Concept Bottleneck Modelscited by 6
  4. 2025Quantifying the Accuracy-Interpretability Trade-Off in Concept-Based Sidechannel Modelscited by 3
  5. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  6. 2025Debugging Concept Bottleneck Models through Removal and Retrainingcited by 2
  7. 2026Mixture of Concept Bottleneck Expertscited by 2
  8. 2026Rethinking Concept Bottleneck Models: From Pitfalls to Solutionscited by 2
  9. 2026Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotationscited by 2
  10. 2026Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretabilitycited by 1
  11. 2025Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodingscited by 0
  12. 2026Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Modelscited by 0
3 more
  1. 2026Federated Concept-Based Models: Interpretable models with distributed supervisioncited by 0
  2. 2026Can VLMs Reason Robustly? A Neuro-Symbolic Investigationcited by 0
  3. 2026CHOQOLATE: Organizing Concept Bottleneck Latent Spaces with Choquet Integralscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Concept Bottleneck Models from Mechanistic Explanations
  2. 2024Explain via Any Concept: Concept Bottleneck Model with Open Vocabulary Concepts
  3. 2026CBX-Bench: A Human-Aligned MLLM Council for Benchmarking Concept Bottleneck Model Explanations
  4. 2025Flexible Concept Bottleneck Model
  5. 2025Partially Shared Concept Bottleneck Models
  6. 2025Enhancing Interpretable Image Classification Through LLM Agents and Conditional Concept Bottleneck Models
  7. 2025V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer
  8. 2025Interpretable Generative Models through Post-hoc Concept Bottlenecks