# Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions

arXiv: https://arxiv.org/abs/2502.06536 · HTML: https://mlsanity.com/paper/2502.06536/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 10 Feb 2025.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Promises and Pitfalls of Black-Box Concept Learning Models](https://mlsanity.com/paper/2106.13314/) (2021)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Causal Consistency of Structural Equation Models](https://mlsanity.com/paper/1707.00819/) (2017)
- [Do Concept Bottleneck Models Learn as Intended?](https://mlsanity.com/paper/2105.04289/) (2021)
- [Multi-View Causal Representation Learning with Partial Observability](https://mlsanity.com/paper/2311.04056/) (2023)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Approximate Causal Abstraction](https://mlsanity.com/paper/1906.11583/) (2019)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies](https://mlsanity.com/paper/2401.04890/) (2024)
- [Unit Testing for Concepts in Neural Networks](https://mlsanity.com/paper/2208.10244/) (2022)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)

## Built on by (most cited first, 6 of 6)

- [If Concept Bottlenecks are the Question, are Foundation Models the Answer?](https://mlsanity.com/paper/2504.19774/) (2025)
- [When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective](https://mlsanity.com/paper/2506.03784/) (2025)
- [Matryoshka Concept Bottleneck Models](https://mlsanity.com/paper/2605.20612/) (2026)
- [Intervening to Learn and Compose Causally Disentangled Representations](https://mlsanity.com/paper/2507.04754/) (2025)
- [Beyond identifiability: Learning causal representations with few environments and finite samples](https://mlsanity.com/paper/2603.25796/) (2026)
- [TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessment](https://mlsanity.com/paper/2606.30313/) (2026)

## Abstract (arXiv)

Machine learning is a vital part of many real-world systems, but several concerns remain about the lack of interpretability, explainability and robustness of black-box AI systems. Concept Bottleneck Models (CBM) address some of these challenges by learning interpretable concepts from high-dimensional data, e.g. images, which are used to predict labels. An important issue in CBMs are spurious correlation between concepts, which effectively lead to learning "wrong" concepts. Current mitigating strategies have strong assumptions, e.g., they assume that the concepts are statistically independent of each other, or require substantial interaction in terms of both interventions and labels provided by annotators. In this paper, we describe a framework that provides theoretical guarantees on the correctness of the learned concepts and on the number of required labels, without requiring any interventions. Our framework leverages causal representation learning (CRL) methods to learn latent causal variables from high-dimensional observations in a unsupervised way, and then learns to align these variables with interpretable concepts with few concept labels. We propose a linear and a non-parametric estimator for this mapping, providing a finite-sample high probability result in the linear case and an asymptotic consistency result for the non-parametric estimator. We evaluate our framework in synthetic and image benchmarks, showing that the learned concepts have less impurities and are often more accurate than other CBMs, even in settings with strong correlations between concepts.