# Unsupervised Causal Binary Concepts Discovery with VAE for Black-box Model Explanation

arXiv: https://arxiv.org/abs/2109.04518 · HTML: https://mlsanity.com/paper/2109.04518/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://mlsanity.com/paper/1709.02023/) (2017)
- [Explaining Classifiers with Causal Concept Effect (CaCE)](https://mlsanity.com/paper/1907.07165/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [CXPlain: Causal Explanations for Model Interpretation under Uncertainty](https://mlsanity.com/paper/1910.12336/) (2019)
- [Generative causal explanations of black-box classifiers](https://mlsanity.com/paper/2006.13913/) (2020)
- [Explaining Deep Learning Models using Causal Inference](https://mlsanity.com/paper/1811.04376/) (2018)
- [Explaining Visual Models by Causal Attribution](https://mlsanity.com/paper/1909.08891/) (2019)
- [Benchmarks, Algorithms, and Metrics for Hierarchical Disentanglement](https://mlsanity.com/paper/2102.05185/) (2021)
- [Learning Interpretable Models with Causal Guarantees](https://mlsanity.com/paper/1901.08576/) (2019)
- [PatchVAE: Learning Local Latent Codes for Recognition](https://mlsanity.com/paper/2004.03623/) (2020)
- [Discond-VAE: Disentangling Continuous Factors from the Discrete](https://mlsanity.com/paper/2009.08039/) (2020)
- [Improving Disentangled Representation Learning with the Beta Bernoulli Process](https://mlsanity.com/paper/1909.01839/) (2019)

## Built on by (most cited first, 8 of 8)

- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [ECLAD: Extracting Concepts with Local Aggregated Descriptors](https://mlsanity.com/paper/2206.04531/) (2022)
- [Deep Causal Learning: Representation, Discovery and Inference](https://mlsanity.com/paper/2211.03374/) (2022)
- [A survey on Concept-based Approaches For Model Improvement](https://mlsanity.com/paper/2403.14566/) (2024)
- [Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs](https://mlsanity.com/paper/2305.18362/) (2023)
- [SurroCBM: Concept Bottleneck Surrogate Models for Generative Post-hoc Explanation](https://mlsanity.com/paper/2310.07698/) (2023)
- [LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multimodal Large Language Models](https://mlsanity.com/paper/2406.14862/) (2024)
- [Towards Verified and Targeted Explanations through Formal Methods](https://mlsanity.com/paper/2604.14209/) (2026)

## Abstract (arXiv)

We aim to explain a black-box classifier with the form: `data X is classified as class Y because X textithas A, B and textitdoes not have C' in which A, B, and C are high-level concepts. The challenge is that we have to discover in an unsupervised manner a set of concepts, i.e., A, B and C, that is useful for the explaining the classifier. We first introduce a structural generative model that is suitable to express and discover such concepts. We then propose a learning process that simultaneously learns the data distribution and encourages certain concepts to have a large causal influence on the classifier output. Our method also allows easy integration of user's prior knowledge to induce high interpretability of concepts. Using multiple datasets, we demonstrate that our method can discover useful binary concepts for explanation.