mlsanityData to 7 Oct 2026

On the Identifiability of Nonlinear ICA: Sparsity and Beyond

arXiv 2206.07751 ↗First posted 15 Jun 2022cs.LG cs.AI stat.MLcited by 84

Not on the map. Why?

Abstract (arXiv)

Nonlinear independent component analysis (ICA) aims to recover the underlying independent latent sources from their observable nonlinear mixtures. How to make the nonlinear ICA model identifiable up to certain trivial indeterminacies is a long-standing problem in unsupervised learning. Recent breakthroughs reformulate the standard independence assumption of sources as conditional independence given some auxiliary variables (e.g., class labels and/or domain/time indexes) as weak supervision or inductive bias. However, nonlinear ICA with unconditional priors cannot benefit from such developments. We explore an alternative path and consider only assumptions on the mixing process, such as Structural Sparsity. We show that under specific instantiations of such constraints, the independent latent sources can be identified from their nonlinear mixtures up to a permutation and a component-wise transformation, thus achieving nontrivial identifiability of nonlinear ICA without auxiliary variables. We provide estimation methods and validate the theoretical results experimentally. The results on image data suggest that our conditions may hold in a number of practical data generating processes.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 17 of its 46 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

17 of 46 references.

References · 17

  1. 2016Density estimation using Real NVPcited by 2,029
  2. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  3. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  4. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  8. 2021Independent mechanism analysis, a new concept?cited by 96
  9. 2021Learning Temporally Causal Latent Processes from General Temporal Datacited by 96
  10. 2020Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)cited by 91
  11. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76
  12. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
5 more
  1. 2021iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compressioncited by 27
  2. 2021I Don't Need u: Identifiable Non-Linear ICA Without Side Informationcited by 21
  3. 2017Adaptive Estimation in Structured Factor Models with Applications to Overlapping Clusteringcited by 19
  4. 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
  5. 2020Vintage Factor Analysis with Varimax Performs Statistical Inferencecited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 16; 2024: 23; 2025: 25; 2026: 17.

84 papers.

Sort:

By area:

  1. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  2. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 72
  3. 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 67
  4. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  5. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  6. 2023Provably Learning Object-Centric Representationscited by 45
  7. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  8. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  9. 2023Disentanglement via Latent Quantizationcited by 40
  10. 2023Understanding Masked Autoencoders via Hierarchical Latent Variable Modelscited by 39
  11. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 34
  12. 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 33
38 more
  1. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 30
  2. 2024Unifying Causal Representation Learning with the Invariance Principlecited by 30
  3. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  4. 2025Thought Communication in Multiagent Collaborationcited by 28
  5. 2023Identification of Nonlinear Latent Hierarchical Modelscited by 27
  6. 2023Learning World Models with Identifiable Factorizationcited by 23
  7. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  8. 2024Learning Discrete Concepts in Latent Hierarchical Modelscited by 17
  9. 2024The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applicationscited by 17
  10. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 15
  11. 2024Counterfactual Generation with Identifiability Guaranteescited by 13
  12. 2024Tripod: Three Complementary Inductive Biases for Disentangled Representation Learningcited by 13
  13. 2024Causal Temporal Representation Learning with Nonstationary Sparse Transitioncited by 12
  14. 2024Causal Representation Learning from Multimodal Biomedical Observationscited by 12
  15. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  16. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  17. 2025DeCaFlow: A deconfounding causal generative modelcited by 8
  18. 2025Towards Identifiability of Hierarchical Temporal Causal Representation Learningcited by 7
  19. 2025Learning General Causal Structures with Hidden Dynamic Process for Climate Analysiscited by 6
  20. 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
  21. 2024Towards the Reusability and Compositionality of Causal Representationscited by 5
  22. 2024On the Identifiability of Sparse ICA without Assuming Non-Gaussianitycited by 5
  23. 2024Causal Representation Learning in Temporal Data via Single-Parent Decodingcited by 5
  24. 2026Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modelingcited by 5
  25. 2024From Orthogonality to Dependency: Learning Disentangled Representation for Multi-Modal Time-Series Sensing Signalscited by 4
  26. 2024A Non-negative VAE:the Generalized Gamma Belief Networkcited by 4
  27. 2024Range, not Independence, Drives Modularity in Biologically Inspired Representationscited by 4
  28. 2024Learning Identifiable Factorized Causal Representations of Cellular Responsescited by 4
  29. 2025Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learningcited by 4
  30. 2025Nonparametric Factor Analysis and Beyondcited by 4
  31. 2025The Third Pillar of Causal Analysis? A Measurement Perspective on Causal Representationscited by 4
  32. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  33. 2025Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiabilitycited by 4
  34. 2026PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICAcited by 4
  35. 2023On the Identifiability of Quantized Factorscited by 3
  36. 2024Differentiable Causal Discovery For Latent Hierarchical Causal Modelscited by 3
  37. 2025Disentangling Long-Short Term State Under Unknown Interventions for Online Time Series Forecastingcited by 3
  38. 2025Mechanistic Independence: A Principle for Identifiable Disentangled Representationscited by 3

50 most cited of 84 All 84 (JSON)