# Sparse Linear Concept Discovery Models

arXiv: https://arxiv.org/abs/2308.10782 · HTML: https://mlsanity.com/paper/2308.10782/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 18 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 10 of 10)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Leveraging Sparse Linear Layers for Debuggable Deep Networks](https://mlsanity.com/paper/2105.04857/) (2021)
- [Nonparametric Bayesian Deep Networks with Local Competition](https://mlsanity.com/paper/1805.07624/) (2018)
- [Contextual Semantic Interpretability](https://mlsanity.com/paper/2009.08720/) (2020)
- [Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representations](https://mlsanity.com/paper/2201.03624/) (2022)

## Built on by (most cited first, 22 of 22)

- [Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)](https://mlsanity.com/paper/2402.10376/) (2024)
- [Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery](https://mlsanity.com/paper/2407.14499/) (2024)
- [Stochastic Concept Bottleneck Models](https://mlsanity.com/paper/2406.19272/) (2024)
- [Coarse-to-Fine Concept Bottleneck Models](https://mlsanity.com/paper/2310.02116/) (2023)
- [Bayesian Concept Bottleneck Models with LLM Priors](https://mlsanity.com/paper/2410.15555/) (2024)
- [DCBM: Data-Efficient Visual Concept Bottleneck Models](https://mlsanity.com/paper/2412.11576/) (2024)
- [CLIP-QDA: An Explainable Concept Bottleneck Model](https://mlsanity.com/paper/2312.00110/) (2023)
- [AdaCBM: An Adaptive Concept Bottleneck Model for Explainable and Accurate Diagnosis](https://mlsanity.com/paper/2408.02001/) (2024)
- [Graph Concept Bottleneck Models](https://mlsanity.com/paper/2508.14255/) (2025)
- [DISCOVER: Making Vision Networks Interpretable via Competition and Dissection](https://mlsanity.com/paper/2310.04929/) (2023)
- [Enhancing Concept Localization in CLIP-based Concept Bottleneck Models](https://mlsanity.com/paper/2510.07115/) (2025)
- [Transformation of audio embeddings into interpretable, concept-based representations](https://mlsanity.com/paper/2504.14076/) (2025)
- [CAPE: CAM as a Probabilistic Ensemble for Enhanced DNN Interpretation](https://mlsanity.com/paper/2404.02388/) (2024)
- [Improving Concept Alignment in Vision-Language Concept Bottleneck Models](https://mlsanity.com/paper/2405.01825/) (2024)
- [Zero-shot Concept Bottleneck Models](https://mlsanity.com/paper/2502.09018/) (2025)
- [Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders](https://mlsanity.com/paper/2506.23951/) (2025)
- [SPADE: Sparsity-Guided Debugging for Deep Neural Networks](https://mlsanity.com/paper/2310.04519/) (2023)
- [IP-CRR: Information Pursuit for Interpretable Classification of Chest Radiology Reports](https://mlsanity.com/paper/2505.00191/) (2025)
- [CLIP-Free, Label Free, Unsupervised Concept Bottleneck Models](https://mlsanity.com/paper/2503.10981/) (2025)
- [Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues](https://mlsanity.com/paper/2602.12381/) (2026)
- [Spatially Grounded Concept-Based Image Classification](https://mlsanity.com/paper/2510.04180/) (2025)
- [Clarity: The Flexibility-Interpretability Trade-Off in Sparsity-aware Concept Bottleneck Models](https://mlsanity.com/paper/2601.21944/) (2026)

## Abstract (arXiv)

The recent mass adoption of DNNs, even in safety-critical scenarios, has shifted the focus of the research community towards the creation of inherently intrepretable models. Concept Bottleneck Models (CBMs) constitute a popular approach where hidden layers are tied to human understandable concepts allowing for investigation and correction of the network's decisions. However, CBMs usually suffer from: (i) performance degradation and (ii) lower interpretability than intended due to the sheer amount of concepts contributing to each decision. In this work, we propose a simple yet highly intuitive interpretable framework based on Contrastive Language Image models and a single sparse linear layer. In stark contrast to related approaches, the sparsity in our framework is achieved via principled Bayesian arguments by inferring concept presence via a data-driven Bernoulli distribution. As we experimentally show, our framework not only outperforms recent CBM approaches accuracy-wise, but it also yields high per example concept sparsity, facilitating the individual investigation of the emerging concepts.