# When Are Concept Bottleneck Model Explanations Faithful and Compact?

arXiv: https://arxiv.org/abs/2610.06285 · HTML: https://mlsanity.com/paper/2610.06285/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 74 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 7 of 7)

- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Do Concept Bottleneck Models Learn as Intended?](https://mlsanity.com/paper/2105.04289/) (2021)
- [How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation](https://mlsanity.com/paper/1802.00682/) (2018)
- [Formal Explanations for Neuro-Symbolic AI](https://mlsanity.com/paper/2410.14219/) (2024)
- [Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations](https://mlsanity.com/paper/2605.16405/) (2026)
- [Prototype Language Models](https://mlsanity.com/paper/2607.00510/) (2026)
- [Are Concept Bottleneck Models Effective as Decision-Support Systems?](https://mlsanity.com/paper/2608.25581/) (2026)

## Abstract (arXiv)

Concept bottleneck models (CBMs) are neural classifiers that allow to explain their decisions via high-level concepts, potentially enabling understanding, steering and debugging. However, their explanations are often derived heuristically. Building on formal explainability, we argue they should also be faithful, i.e., not misreport which concepts actually matter. We show that, for widespread CBM architectures, including recent VLM-based variants, faithful explanations must include all concepts in the bottleneck, compromising interpretability when this is large. This result applies to both heuristic and faithful-by-construction formal explanations. To encourage the existence of compact faithful explanations, we suggest i) modeling concepts probabilistically as binary or categorical random variables (rather than logits), and ii) employing per-concept training-time sparsification via group lasso (rather than regular elastic net). We also extend algorithms from formal explainability to CBMs, and show they outperform natural heuristics in terms of guarantees and explanation size. Overall, our work warns against naive interpretability claims and provides formal conditions and practical strategies for ensuring CBMs are as interpretable as advertised.