# Towards a Deeper Understanding of Concept Bottleneck Models Through End-to-End Explanation

arXiv: https://arxiv.org/abs/2302.03578 · HTML: https://mlsanity.com/paper/2302.03578/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 9 of 9)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Do Concept Bottleneck Models Learn as Intended?](https://mlsanity.com/paper/2105.04289/) (2021)

## Built on by (most cited first, 7 of 7)

- [On the Concept Trustworthiness in Concept Bottleneck Models](https://mlsanity.com/paper/2403.14349/) (2024)
- [If Concept Bottlenecks are the Question, are Foundation Models the Answer?](https://mlsanity.com/paper/2504.19774/) (2025)
- [Do Concept Bottleneck Models Respect Localities?](https://mlsanity.com/paper/2401.01259/) (2024)
- [Can we Constrain Concept Bottleneck Models to Learn Semantically Meaningful Input Features?](https://mlsanity.com/paper/2402.00912/) (2024)
- [Rethinking Distance Metrics for Counterfactual Explainability](https://mlsanity.com/paper/2410.14522/) (2024)
- [SL-CBM: Enhancing Concept Bottleneck Models with Semantic Locality for Better Interpretability](https://mlsanity.com/paper/2601.12804/) (2026)
- [Towards Robust and Reliable Concept Representations: Reliability-Enhanced Concept Embedding Model](https://mlsanity.com/paper/2502.01191/) (2025)

## Abstract (arXiv)

Concept Bottleneck Models (CBMs) first map raw input(s) to a vector of human-defined concepts, before using this vector to predict a final classification. We might therefore expect CBMs capable of predicting concepts based on distinct regions of an input. In doing so, this would support human interpretation when generating explanations of the model's outputs to visualise input features corresponding to concepts. The contribution of this paper is threefold: Firstly, we expand on existing literature by looking at relevance both from the input to the concept vector, confirming that relevance is distributed among the input features, and from the concept vector to the final classification where, for the most part, the final classification is made using concepts predicted as present. Secondly, we report a quantitative evaluation to measure the distance between the maximum input feature relevance and the ground truth location; we perform this with the techniques, Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG) and a baseline gradient approach, finding LRP has a lower average distance than IG. Thirdly, we propose using the proportion of relevance as a measurement for explaining concept importance.