# Coarse-to-Fine Concept Bottleneck Models

arXiv: https://arxiv.org/abs/2310.02116 · HTML: https://mlsanity.com/paper/2310.02116/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks](https://mlsanity.com/paper/2204.10965/) (2022)
- [Probabilistic Concept Bottleneck Models](https://mlsanity.com/paper/2306.01574/) (2023)
- [Leveraging Sparse Linear Layers for Debuggable Deep Networks](https://mlsanity.com/paper/2105.04857/) (2021)
- [Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability](https://mlsanity.com/paper/2207.09615/) (2022)
- [Sparse Linear Concept Discovery Models](https://mlsanity.com/paper/2308.10782/) (2023)
- [Nonparametric Bayesian Deep Networks with Local Competition](https://mlsanity.com/paper/1805.07624/) (2018)
- [Contextual Semantic Interpretability](https://mlsanity.com/paper/2009.08720/) (2020)

## Built on by (most cited first, 30 of 31)

- [Stochastic Concept Bottleneck Models](https://mlsanity.com/paper/2406.19272/) (2024)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Object Centric Concept Bottlenecks](https://mlsanity.com/paper/2505.24492/) (2025)
- [PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck](https://mlsanity.com/paper/2403.05297/) (2024)
- [SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions](https://mlsanity.com/paper/2507.23784/) (2025)
- [Enhancing Concept Localization in CLIP-based Concept Bottleneck Models](https://mlsanity.com/paper/2510.07115/) (2025)
- [An Analysis of Concept Bottleneck Models: Measuring, Understanding, and Mitigating the Impact of Noisy Annotations](https://mlsanity.com/paper/2505.16705/) (2025)
- [Partially Shared Concept Bottleneck Models](https://mlsanity.com/paper/2511.22170/) (2025)
- [MaskSDM with Shapley values to improve flexibility, robustness, and explainability in species distribution modeling](https://mlsanity.com/paper/2503.13057/) (2025)
- [Hierarchical Concept-based Interpretable Models](https://mlsanity.com/paper/2602.23947/) (2026)
- [Restyling Unsupervised Concept Based Interpretable Networks with Generative Models](https://mlsanity.com/paper/2407.01331/) (2024)
- [Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations](https://mlsanity.com/paper/2510.17256/) (2025)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [Routesplain: Towards Faithful and Intervenable Routing for Software-related Tasks](https://mlsanity.com/paper/2511.09373/) (2025)
- [MAIN-VLA: Modeling Abstraction of Intention and eNvironment for Vision-Language-Action Models](https://mlsanity.com/paper/2602.02212/) (2026)
- [COCO-Tree: Compositional Hierarchical Concept Trees for Enhanced Reasoning in Vision Language Models](https://mlsanity.com/paper/2510.11012/) (2025)
- [Hierarchical, Interpretable, Label-Free Concept Bottleneck Model](https://mlsanity.com/paper/2604.02468/) (2026)
- [Hoeffding Concept Bottleneck Models with Applications to Overhead Images](https://mlsanity.com/paper/2606.00082/) (2026)
- [Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings](https://mlsanity.com/paper/2507.07532/) (2025)
- [Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Models](https://mlsanity.com/paper/2509.17522/) (2025)
- [Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models](https://mlsanity.com/paper/2601.21944/) (2026)
- [Hyperbolic Concept Bottleneck Models](https://mlsanity.com/paper/2605.06440/) (2026)
- [ConceptM3oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology](https://mlsanity.com/paper/2605.24399/) (2026)
- [When Interpretability Becomes a Liability: Adversarial Attacks on CBM Concept Layers](https://mlsanity.com/paper/2605.25304/) (2026)
- [Bias Leaves a Gradient Trail: Label-Free Bias Identification via Gradient Probes on Concept Decompositions](https://mlsanity.com/paper/2605.28780/) (2026)
- [Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models](https://mlsanity.com/paper/2606.04326/) (2026)
- [Formal Concept Lattices are Good Semantic Scaffolds for Concept-Based Learning](https://mlsanity.com/paper/2606.05471/) (2026)
- [In Defense of Information Leakage in Concept-based Models](https://mlsanity.com/paper/2606.10669/) (2026)
- [COCOLogic-V2: Identifying Logical Inconsistencies via Truly Hard-Negatives](https://mlsanity.com/paper/2606.28194/) (2026)
- [CHOQOLATE: Organizing Concept Bottleneck Latent Spaces with Choquet Integrals](https://mlsanity.com/paper/2609.37786/) (2026)

## Abstract (arXiv)

Deep learning algorithms have recently gained significant attention due to their impressive performance. However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. This work targets ante hoc interpretability, and specifically Concept Bottleneck Models (CBMs). Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on two levels of granularity. To this end, we propose a novel two-level concept discovery formulation leveraging: (i) recent advances in vision-language models, and (ii) an innovative formulation for coarse-to-fine concept selection via data-driven and sparsity-inducing Bayesian arguments. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the proposed construction not only outperforms recent CBM approaches, but also yields a principled framework towards interpetability.