mlsanityData to 14 Oct 2026

Learning Linear Causal Representations from Interventions under General Nonlinear Mixing

arXiv 2306.02235 ↗First posted 4 Jun 2023cs.LG cs.AI math.STcited by 92

Not on the map. Why?

Abstract (arXiv)

We study the problem of learning causal representations from unknown, latent interventions in a general setting, where the latent distribution is Gaussian but the mixing function is completely general. We prove strong identifiability results given unknown single-node interventions, i.e., without having access to the intervention targets. This generalizes prior works which have focused on weaker classes, such as linear maps or paired counterfactual data. This is also the first instance of causal identifiability from non-paired interventions for deep neural network embeddings. Our proof relies on carefully uncovering the high-dimensional geometric structure present in the data distribution after a non-linear density transformation, which we capture by analyzing quadratic forms of precision matrices of the latent distributions. Finally, we propose a contrastive algorithm to identify the latent variables in practice and evaluate its performance on various tasks.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Builds on 65 of its 107 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

65 of 107 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  10. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  11. 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 752
  12. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 536
38 more
  1. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  2. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  3. 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 331
  4. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  7. 2020The Risks of Invariant Risk Minimizationcited by 262
  8. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  9. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  10. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 220
  11. 2021Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Gamecited by 161
  12. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
  13. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
  14. 2022Interventional Causal Representation Learningcited by 142
  15. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  16. 2022Weakly supervised causal representation learningcited by 140
  17. 2022DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterizationcited by 140
  18. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  19. 2018Anchor regression: heterogeneous data meets causalitycited by 118
  20. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  21. 2020On Linear Identifiability of Learned Representationscited by 105
  22. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  23. 2021Independent mechanism analysis, a new concept?cited by 97
  24. 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
  25. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 85
  26. 2022Linear Causal Disentanglement via Interventionscited by 84
  27. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  28. 2019Permutation-Based Causal Structure Learning with Unknown Intervention Targetscited by 69
  29. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  30. 2020Weakly Supervised Disentangled Generative Causal Representation Learningcited by 68
  31. 2022Weakly Supervised Representation Learning with Sparse Perturbationscited by 65
  32. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  33. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  34. 2023Causal Component Analysiscited by 56
  35. 2021Learning latent causal graphs via mixture oraclescited by 55
  36. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  37. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  38. 2023Score-based Causal Representation Learning with Interventionscited by 49

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 25; 2025: 26; 2026: 25.

92 papers.

Sort:

By area:

  1. 2023Identifiability Guarantees for Causal Disentanglement from Soft Interventionscited by 84
  2. 2024On the Origins of Linear Representations in Large Language Modelscited by 79
  3. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 74
  4. 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 68
  5. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 58
  6. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 37
  7. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 35
  8. 2023Learning nonparametric latent causal graphs with unknown interventionscited by 34
  9. 2023General Identifiability and Achievability for Causal Representation Learningcited by 34
  10. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 32
  11. 2024Unifying Causal Representation Learning with the Invariance Principlecited by 31
  12. 2026When Does LeJEPA Learn a World Model?cited by 30
38 more
  1. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  2. 2023Identification of Nonlinear Latent Hierarchical Modelscited by 27
  3. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  4. 2023Identifiable Latent Polynomial Causal Models Through the Lens of Changecited by 23
  5. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 23
  6. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 22
  7. 2023Identifying Representations for Intervention Extrapolationcited by 21
  8. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  9. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 20
  10. 2023Identifying Linearly-Mixed Causal Representations from Multi-Node Interventionscited by 18
  11. 2024Linear Causal Representation Learning from Unknown Multi-node Interventionscited by 14
  12. 2025SmartCLIP: Modular Vision-language Alignment with Identification Guaranteescited by 14
  13. 2024Causal Representation Learning from Multimodal Biomedical Observationscited by 13
  14. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  15. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 13
  16. 2024Smoke and Mirrors in Causal Downstream Taskscited by 12
  17. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  18. 2024Beyond DAGs: A Latent Partial Causal Model for Multimodal Learningcited by 10
  19. 2023Deep Backtracking Counterfactuals for Causally Compliant Explanationscited by 9
  20. 2023An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysiscited by 9
  21. 2024Language Agents Meet Causality -- Bridging LLMs and Causal World Modelscited by 9
  22. 2024Automated Discovery of Pairwise Interactions from Unstructured Datacited by 8
  23. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  24. 2024Identifiability Guarantees for Causal Disentanglement from Purely Observational Datacited by 7
  25. 2025Robustness of Nonlinear Representation Learningcited by 7
  26. 2025From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?cited by 7
  27. 2024Propensity Score Alignment of Unpaired Multimodal Datacited by 6
  28. 2025What is causal about causal models and representations?cited by 6
  29. 2023Invariance & Causal Representation Learning: Prospects and Limitationscited by 5
  30. 2024Towards Identifiable Latent Additive Noise Modelscited by 5
  31. 2025Sanity Checking Causal Representation Learning on a Simple Real-World Systemcited by 5
  32. 2026Causal Representation Learning from General Environments under Nonparametric Mixingcited by 5
  33. 2024Toward the Identifiability of Comparative Deep Generative Modelscited by 4
  34. 2024The Causal Information Bottleneck and Optimal Causal Variable Abstractionscited by 4
  35. 2025The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representationscited by 4
  36. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  37. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4
  38. 2025CausalVerse: Benchmarking Causal Representation Learning with Configurable High-Fidelity Simulationscited by 4

50 most cited of 92 All 92 (JSON)