# When are Post-hoc Conceptual Explanations Identifiable?

arXiv: https://arxiv.org/abs/2206.13872 · HTML: https://mlsanity.com/paper/2206.13872/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 62 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [On Disentangled Representations Learned From Correlated Data](https://mlsanity.com/paper/2006.07886/) (2020)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [Now You See Me (CME): Concept-based Model Extraction](https://mlsanity.com/paper/2010.13233/) (2020)

## Built on by (most cited first, 14 of 14)

- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces](https://mlsanity.com/paper/2212.14855/) (2022)
- [Interaction Asymmetry: A General Principle for Learning Composable Abstractions](https://mlsanity.com/paper/2411.07784/) (2024)
- [Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?](https://mlsanity.com/paper/2409.13456/) (2024)
- [Uncovering Meanings of Embeddings via Partial Orthogonality](https://mlsanity.com/paper/2310.17611/) (2023)
- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)
- [Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces](https://mlsanity.com/paper/2311.14435/) (2023)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Nonparametric Identification of Latent Concepts](https://mlsanity.com/paper/2510.00136/) (2025)
- [The Language of Trauma: Modeling Traumatic Event Descriptions Across Domains with Explainable AI](https://mlsanity.com/paper/2408.05977/) (2024)
- [Intervening to Learn and Compose Causally Disentangled Representations](https://mlsanity.com/paper/2507.04754/) (2025)
- [Are We Merely Justifying Results ex Post Facto? Quantifying Explanatory Inversion in Post-Hoc Model Explanations](https://mlsanity.com/paper/2504.08919/) (2025)
- [Concept Component Analysis: A Principled Approach for Concept Extraction in LLMs](https://mlsanity.com/paper/2601.20420/) (2026)
- [Measuring Explainer Stability via Attribution Separability](https://mlsanity.com/paper/2608.02697/) (2026)

## Abstract (arXiv)

Interest in understanding and factorizing learned embedding spaces through conceptual explanations is steadily growing. When no human concept labels are available, concept discovery methods search trained embedding spaces for interpretable concepts like object shape or color that can provide post-hoc explanations for decisions. Unlike previous work, we argue that concept discovery should be identifiable, meaning that a number of known concepts can be provably recovered to guarantee reliability of the explanations. As a starting point, we explicitly make the connection between concept discovery and classical methods like Principal Component Analysis and Independent Component Analysis by showing that they can recover independent concepts under non-Gaussian distributions. For dependent concepts, we propose two novel approaches that exploit functional compositionality properties of image-generating processes. Our provably identifiable concept discovery methods substantially outperform competitors on a battery of experiments including hundreds of trained models and dependent concepts, where they exhibit up to 29 % better alignment with the ground truth. Our results highlight the strict conditions under which reliable concept discovery without human labels can be guaranteed and provide a formal foundation for the domain. Our code is available online.