mlsanityData to 14 Oct 2026

Towards Interpretable Deep Generative Models via Causal Representation Learning

arXiv 2504.11609 ↗First posted 15 Apr 2025stat.ML cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Recent developments in generative artificial intelligence (AI) rely on machine learning techniques such as deep learning and generative modeling to achieve state-of-the-art performance across wide-ranging domains. These methods' surprising performance is due in part to their ability to learn implicit "representations" of complex, multi-modal data. Unfortunately, deep neural networks are notoriously black boxes that obscure these representations, making them difficult to interpret or analyze. To resolve these difficulties, one approach is to build new interpretable neural network models from the ground up. This is the goal of the emerging field of causal representation learning (CRL) that uses causality as a vector for building flexible, interpretable, and transferable generative AI. CRL can be seen as a synthesis of three intrinsically statistical ideas: (i) latent variable models such as factor analysis; (ii) causal graphical models with latent variables; and (iii) nonparametric statistics and deep learning. This paper introduces CRL from a statistical perspective, focusing on connections to classical models as well as statistical and causal identifiability results. We also highlights key application areas, implementation strategies, and open statistical questions.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 79 of its 154 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

79 of 154 references.

References · 79

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2020Concept Bottleneck Modelscited by 1,032
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  6. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
  7. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  8. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  9. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  10. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  11. 2013Black Box Variational Inferencecited by 319
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
38 more
  1. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 307
  2. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  3. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  4. 2019Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changescited by 166
  5. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
  6. 2022Interventional Causal Representation Learningcited by 142
  7. 2022Weakly supervised causal representation learningcited by 140
  8. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  9. 2018Invariance, Causality and Robustnesscited by 123
  10. 2022Score matching enables causal discovery of nonlinear additive noise modelscited by 111
  11. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  12. 2018The Blessings of Multiple Causescited by 104
  13. 2021Independent mechanism analysis, a new concept?cited by 97
  14. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  15. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 85
  16. 2013Deep Learning of Representations: Looking Forwardcited by 84
  17. 2022Linear Causal Disentanglement via Interventionscited by 84
  18. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  19. 2024Robust agents learn causal world modelscited by 81
  20. 2022Impossibility Theorems for Feature Attributioncited by 78
  21. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  22. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 74
  23. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  24. 2019Permutation-Based Causal Structure Learning with Unknown Intervention Targetscited by 69
  25. 2020Weakly Supervised Disentangled Generative Causal Representation Learningcited by 68
  26. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 65
  27. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  28. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 61
  29. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  30. 2021Learning latent causal graphs via mixture oraclescited by 55
  31. 2022Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesiscited by 55
  32. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  33. 2013Causal Discovery from Changescited by 52
  34. 2022Latent Hierarchical Causal Structure Discovery with Rank Constraintscited by 51
  35. 2023Score-based Causal Representation Learning with Interventionscited by 49
  36. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  37. 2017Nested Markov Properties for Acyclic Directed Mixed Graphscited by 39
  38. 2023BISCUIT: Causal Representation Learning from Binary Interactionscited by 38

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 6.

13 papers.

Sort:

By area:

  1. 2025Thought Communication in Multiagent Collaborationcited by 30
  2. 2025Incorporating Hierarchical Semantics in Sparse Autoencoder Architecturescited by 14
  3. 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
  4. 2025Generative Models for Crystalline Materialscited by 4
  5. 2026Nonlinear multi-study sparse factor analysiscited by 1
  6. 2026Multi-Domain Empirical Bayes for Linearly-Mixed Causal Representationscited by 1
  7. 2026Beyond identifiability: Learning causal representations with few environments and finite samplescited by 1
  8. 2025On Theoretical Identifiability of Discrete Latent Causal Graphical Modelscited by 0
  9. 2025Eigenfunction Extraction for Ordered Representation Learningcited by 0
  10. 2025Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimalitycited by 0
  11. 2026Diverse Dictionary Learningcited by 0
  12. 2026Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applicationscited by 0
1 more
  1. 2026Causal Representation Learning with Instantaneous and Lagged Relations via Nonstationaritycited by 0