mlsanityData to 7 Oct 2026

There Was Never a Bottleneck in Concept Bottleneck Models

arXiv 2506.04877 ↗First posted 5 Jun 2025cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Deep learning representations are often difficult to interpret, which can hinder their deployment in sensitive applications. Concept Bottleneck Models (CBMs) have emerged as a promising approach to mitigate this issue by learning representations that support target task performance while ensuring that each component predicts a concrete concept from a predefined set. In this work, we argue that CBMs do not impose a true bottleneck: the fact that a component can predict a concept does not guarantee that it encodes only information about that concept. This shortcoming raises concerns regarding interpretability and the validity of intervention procedures. To overcome this limitation, we propose Minimal Concept Bottleneck Models (MCBMs), which incorporate an Information Bottleneck (IB) objective to constrain each representation component to retain only the information relevant to its corresponding concept. This IB is implemented via a variational regularization term added to the training loss. As a result, MCBMs yield more interpretable representations, support principled concept-level interventions, and remain consistent with probability-theoretic foundations.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

43 of 60 references.

References · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  5. 2019Invariant Risk Minimizationcited by 1,832
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  7. 2020Concept Bottleneck Modelscited by 1,022
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  11. 2018Disentangling by Factorisingcited by 614
  12. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
31 more
  1. 2019On Mutual Information Maximization for Representation Learningcited by 315
  2. 2023Label-Free Concept Bottleneck Modelscited by 282
  3. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  4. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  5. 2017Emergence of Invariance and Disentanglement in Deep Representationscited by 213
  6. 2020A Theory of Usable Information Under Computational Constraintscited by 153
  7. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 131
  8. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 128
  9. 2021Do Concept Bottleneck Models Learn as Intended?cited by 114
  10. 2018Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustnesscited by 108
  11. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 106
  12. 2018Learning Deep Disentangled Embeddings with the F-Statistic Losscited by 101
  13. 2019On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Datasetcited by 100
  14. 2020The Conditional Entropy Bottleneckcited by 75
  15. 2022GlanceNets: Interpretabile, Leak-proof Concept-based Modelscited by 73
  16. 2023To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Reviewcited by 72
  17. 2024VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidancecited by 65
  18. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 63
  19. 2024Stochastic Concept Bottleneck Modelscited by 49
  20. 2023Learning to Receive Help: Intervention-Aware Concept Embedding Modelscited by 41
  21. 2020CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysiscited by 40
  22. 2024Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretationscited by 33
  23. 2023Towards Robust Metrics for Concept Representation Evaluationcited by 29
  24. 2024Semi-supervised Concept Bottleneck Modelscited by 21
  25. 2019Relevance Factor VAE: Learning and Identifying Disentangled Factorscited by 20
  26. 2021Explainable Deep Learning in Healthcare: A Methodological Survey from an Attribution Viewcited by 15
  27. 2025If Concept Bottlenecks are the Question, are Foundation Models the Answer?cited by 15
  28. 2025Leakage and Interpretability in Concept-Based Modelscited by 13
  29. 2023Financial Time-Series Forecasting: Towards Synergizing Performance And Interpretability Within a Hybrid Machine Learning Approachcited by 10
  30. 2025Measuring Leakage in Concept-Based Methods: An Information Theoretic Approachcited by 9
  31. 2021Interpretability in deep learning for finance: a case study for the Heston modelcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026In-Context Learning Operates as Concept Subspace Learningcited by 3
  2. 2026Exploring Concept Subspace for Self-explainable Text-Attributed Graph Learningcited by 0
  3. 2026In Defense of Information Leakage in Concept-based Modelscited by 0
  4. 2026S2COPE: Self-Supervised Concept Discovery via Preference Learningcited by 0
  5. 2026Assessing Reliability of Symbol Detection in Concept Bottleneck Modelscited by 0
  6. 2026On the Faithfulness of Post-Hoc Concept Bottleneck Modelscited by 0
  7. 2026Loss Invariance Determines What Concept Layers Encode: Volume Grounding in Echocardiographycited by 0