mlsanityData to 14 Oct 2026

Identifiability of latent-variable and structural-equation models: from linear to nonlinear

arXiv 2302.02672 ↗First posted 6 Feb 2023stat.ML cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

An old problem in multivariate statistics is that linear Gaussian models are often unidentifiable, i.e. some parameters cannot be uniquely estimated. In factor (component) analysis, an orthogonal rotation of the factors is unidentifiable, while in linear regression, the direction of effect cannot be identified. For such linear models, non-Gaussianity of the (latent) variables has been shown to provide identifiability. In the case of factor analysis, this leads to independent component analysis, while in the case of the direction of effect, non-Gaussian versions of structural equation modelling solve the problem. More recently, we have shown how even general nonparametric nonlinear versions of such models can be estimated. Non-Gaussianity is not enough in this case, but assuming we have time series, or that the distributions are suitably modulated by some observed auxiliary variables, the models are identifiable. This paper reviews the identifiability theory for the linear and nonlinear cases, considering both factor analytic models and structural equation models.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 36 of its 100 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

36 of 100 references.

References · 36

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  6. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  7. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  8. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  9. 2013Causal Discovery with Continuous Additive Noise Modelscited by 255
  10. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  11. 2018Neural Autoregressive Flowscited by 208
  12. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
24 more
  1. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 122
  2. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  3. 2020Causal Autoregressive Flowscited by 99
  4. 2021Independent mechanism analysis, a new concept?cited by 97
  5. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  6. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 77
  7. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  8. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  9. 2022On the Identifiability and Estimation of Causal Location-Scale Noise Modelscited by 56
  10. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  11. 2015A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extractioncited by 46
  12. 2022Identifying Patient-Specific Root Causes with the Heteroscedastic Noise Modelcited by 38
  13. 2021Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generationcited by 35
  14. 2020On Implicit Regularization in β-VAEscited by 34
  15. 2013ParceLiNGAM: A causal ordering method robust against latent confounderscited by 29
  16. 2021I Don't Need u: Identifiable Non-Linear ICA Without Side Informationcited by 21
  17. 2020Independent Innovation Analysis for Nonlinear Vector Autoregressive Processcited by 19
  18. 2020Relative gradient optimization of the Jacobian term in unsupervised deep learningcited by 16
  19. 2021Structure Learning for Directed Treescited by 9
  20. 2022Identifying Patient-Specific Root Causes of Diseasecited by 8
  21. 2018A Unified Probabilistic Model for Learning Latent Factors and Their Connectivities from High-Dimensional Datacited by 4
  22. 2021Nonlinear Independent Component Analysis for Discrete-Time and Continuous-Time Signalscited by 4
  23. 2021Binary Independent Component Analysis: A Non-stationarity-based Approachcited by 4
  24. 2019Robust contrastive learning and nonlinear ICA in the presence of outlierscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 13; 2025: 21; 2026: 14.

58 papers.

Sort:

By area:

  1. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  2. 2024On the Origins of Linear Representations in Large Language Modelscited by 79
  3. 2023Causal Component Analysiscited by 56
  4. 2025The Origins of Representation Manifolds in Large Language Modelscited by 51
  5. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  6. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 37
  7. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 31
  8. 2025Thought Communication in Multiagent Collaborationcited by 30
  9. 2026When Does LeJEPA Learn a World Model?cited by 30
  10. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  11. 2026There Will Be a Scientific Theory of Deep Learningcited by 29
  12. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 21
38 more
  1. 2024Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structurescited by 20
  2. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 16
  3. 2024Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learningcited by 16
  4. 2025From superposition to sparse codes: interpretable representations in neural networkscited by 16
  5. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  6. 2023Identifying Semantic Component for Robust Molecular Property Predictioncited by 10
  7. 2023Deep Backtracking Counterfactuals for Causally Compliant Explanationscited by 9
  8. 2023Identifying Interpretable Visual Features in Artificial and Biological Neural Systemscited by 9
  9. 2023An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysiscited by 9
  10. 2026On the Identifiability of Controlled World Modelscited by 9
  11. 2024Rethinking Fair Representation Learning for Performance-Sensitive Taskscited by 8
  12. 2023Independent Mechanism Analysis and the Manifold Hypothesiscited by 7
  13. 2024Causal Effect Identification in LiNGAM Models with Latent Confounderscited by 7
  14. 2025Robustness of Nonlinear Representation Learningcited by 7
  15. 2025Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventionscited by 6
  16. 2025Counterfactual Identifiability via Dynamic Optimal Transportcited by 6
  17. 2024From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signalscited by 4
  18. 2025Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learningcited by 4
  19. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  20. 2023A Causal Ordering Prior for Unsupervised Representation Learningcited by 3
  21. 2025Disentangling Long-Short Term State Under Unknown Interventions for Online Time Series Forecastingcited by 3
  22. 2025Flow Stochastic Segmentation Networkscited by 3
  23. 2025Nonparametric Identification of Latent Conceptscited by 3
  24. 2026Score-Based Causal Discovery of Latent Variable Causal Modelscited by 3
  25. 2024Nonstationary Time Series Forecasting via Unknown Distribution Adaptationcited by 2
  26. 2024Analyzing Generative Models by Manifold Entropic Metricscited by 2
  27. 2025Online Time Series Forecasting with Theoretical Guaranteescited by 2
  28. 2025Probabilistic Modelling is Sufficient for Causal Inferencecited by 2
  29. 2025Time Series Domain Adaptation via Latent Invariant Causal Mechanismcited by 1
  30. 2025Causal View of Time Series Imputation: Some Identification Results on Missing Mechanismcited by 1
  31. 2025A Primer on Causal and Statistical Dataset Biases for Fair and Robust Image Analysiscited by 1
  32. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  33. 2026Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixingcited by 1
  34. 2026Learning with Shallow Neural Networks on Cluster-Structured Featurescited by 1
  35. 2024Debiased Model-based Interactive Recommendationcited by 0
  36. 2025Temporal Causal Reasoning with (Non-Recursive) Structural Equation Modelscited by 0
  37. 2025Learning Treatment Representations for Downstream Instrumental Variable Regressioncited by 0
  38. 2025Incorporating Interventional Independence Improves Robustness against Interventional Distribution Shiftcited by 0

50 most cited of 58 All 58 (JSON)