# Towards Interpretable Deep Generative Models via Causal Representation Learning

arXiv: https://arxiv.org/abs/2504.11609 · HTML: https://mlsanity.com/paper/2504.11609/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 79 of its 154 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 30 of 79)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Interventional Causal Representation Learning](https://mlsanity.com/paper/2209.11924/) (2022)
- [Weakly supervised causal representation learning](https://mlsanity.com/paper/2203.16437/) (2022)
- [CITRIS: Causal Identifiability from Temporal Intervened Sequences](https://mlsanity.com/paper/2202.03169/) (2022)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Score matching enables causal discovery of nonlinear additive noise models](https://mlsanity.com/paper/2203.04413/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Independent mechanism analysis, a new concept?](https://mlsanity.com/paper/2106.05200/) (2021)
- [Learning Linear Causal Representations from Interventions under General Nonlinear Mixing](https://mlsanity.com/paper/2306.02235/) (2023)
- [On the Identifiability of Nonlinear ICA: Sparsity and Beyond](https://mlsanity.com/paper/2206.07751/) (2022)
- [Deep Learning of Representations: Looking Forward](https://mlsanity.com/paper/1305.0445/) (2013)
- [Linear Causal Disentanglement via Interventions](https://mlsanity.com/paper/2211.16467/) (2022)
- [Identifiability Guarantees for Causal Disentanglement from Soft Interventions](https://mlsanity.com/paper/2307.06250/) (2023)

## Built on by (most cited first, 13 of 13)

- [Thought Communication in Multiagent Collaboration](https://mlsanity.com/paper/2510.20733/) (2025)
- [Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures](https://mlsanity.com/paper/2506.01197/) (2025)
- [Debiasing Reward Models by Representation Learning with Guarantees](https://mlsanity.com/paper/2510.23751/) (2025)
- [Generative Models for Crystalline Materials](https://mlsanity.com/paper/2511.22652/) (2025)
- [Nonlinear multi-study sparse factor analysis](https://mlsanity.com/paper/2601.18128/) (2026)
- [Multi-Domain Empirical Bayes for Linearly-Mixed Causal Representations](https://mlsanity.com/paper/2603.18404/) (2026)
- [Beyond identifiability: Learning causal representations with few environments and finite samples](https://mlsanity.com/paper/2603.25796/) (2026)
- [On Theoretical Identifiability of Discrete Latent Causal Graphical Models](https://mlsanity.com/paper/2505.18410/) (2025)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)
- [Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality](https://mlsanity.com/paper/2512.10720/) (2025)
- [Diverse Dictionary Learning](https://mlsanity.com/paper/2604.17568/) (2026)
- [Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applications](https://mlsanity.com/paper/2606.06288/) (2026)
- [Causal Representation Learning with Instantaneous and Lagged Relations via Nonstationarity](https://mlsanity.com/paper/2610.03452/) (2026)

## Abstract (arXiv)

Recent developments in generative artificial intelligence (AI) rely on machine learning techniques such as deep learning and generative modeling to achieve state-of-the-art performance across wide-ranging domains. These methods' surprising performance is due in part to their ability to learn implicit "representations" of complex, multi-modal data. Unfortunately, deep neural networks are notoriously black boxes that obscure these representations, making them difficult to interpret or analyze. To resolve these difficulties, one approach is to build new interpretable neural network models from the ground up. This is the goal of the emerging field of causal representation learning (CRL) that uses causality as a vector for building flexible, interpretable, and transferable generative AI. CRL can be seen as a synthesis of three intrinsically statistical ideas: (i) latent variable models such as factor analysis; (ii) causal graphical models with latent variables; and (iii) nonparametric statistics and deep learning. This paper introduces CRL from a statistical perspective, focusing on connections to classical models as well as statistical and causal identifiability results. We also highlights key application areas, implementation strategies, and open statistical questions.