mlsanityData to 7 Oct 2026

Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning

arXiv 2303.16535 ↗First posted 29 Mar 2023cs.LG stat.MLcited by 48

Not on the map. Why?

Abstract (arXiv)

A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement". Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e., based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic due to the lack of identifiability, i.e., uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state-of-the-art of nonlinear ICA theory and algorithms.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 46 of its 111 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

46 of 111 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  6. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  7. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  8. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  9. 2018Disentangling by Factorisingcited by 614
  10. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  11. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  12. 2017Look, Listen and Learncited by 452
34 more
  1. 2018Understanding disentangling in β-VAEcited by 332
  2. 2015A note on the evaluation of generative modelscited by 291
  3. 2020A Survey on Neural Network Interpretabilitycited by 291
  4. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  5. 2021How to Train Your Energy-Based Modelscited by 271
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 246
  8. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 240
  9. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  10. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  11. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  12. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 131
  13. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 127
  14. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  15. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  16. 2022Learnable latent embeddings for joint behavioral and neural analysiscited by 113
  17. 2020Uncovering the structure of clinical EEG signals with self-supervised learningcited by 109
  18. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  19. 2021Independent mechanism analysis, a new concept?cited by 96
  20. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  21. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76
  22. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  23. 2018Structured Disentangled Representationscited by 62
  24. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  25. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  26. 2022Function Classes for Identifiable Nonlinear Independent Component Analysiscited by 53
  27. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  28. 2014Unsupervised Learning of Spatiotemporally Coherent Metricscited by 45
  29. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  30. 2020Independent Innovation Analysis for Nonlinear Vector Autoregressive Processcited by 18
  31. 2018Auto-Encoding Total Correlation Explanationcited by 17
  32. 2020Relative gradient optimization of the Jacobian term in unsupervised deep learningcited by 16
  33. 2022On Finite-Sample Identifiability of Contrastive Learning-Based Nonlinear Independent Component Analysiscited by 7
  34. 2015Rethinking LDA: moment matching for discrete ICAcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 11; 2025: 14; 2026: 16.

48 papers.

Sort:

By area:

  1. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  2. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  3. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  4. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  5. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  6. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 12
  7. 2024The Causal Chambers: Real Physical Systems as a Testbed for AI Methodologycited by 12
  8. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  9. 2024Self-supervised contrastive learning performs non-linear system identificationcited by 9
  10. 2025Learning General Causal Structures with Hidden Dynamic Process for Climate Analysiscited by 6
  11. 2025Quantifying Structure in CLIP Embeddings: A Statistical Framework for Concept Interpretationcited by 6
  12. 2025Debiasing Reward Models by Representation Learning with Guaranteescited by 6
36 more
  1. 2023Identifiable Feature Learning for Spatial Data with Nonlinear ICAcited by 4
  2. 2024An Independence-promoting Loss for Music Generation with Language Modelscited by 4
  3. 2025Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learningcited by 4
  4. 2025Connecting Neural Models Latent Geometries with Relative Geodesic Representationscited by 4
  5. 2026PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICAcited by 4
  6. 2026Understanding Self-Supervised Learning via Latent Distribution Matchingcited by 4
  7. 2024Unsupervised Discovery of Clinical Disease Signatures Using Probabilistic Independencecited by 3
  8. 2026Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attentioncited by 3
  9. 2024Unsupervised Representation Learning from Sparse Transformation Analysiscited by 2
  10. 2024Analyzing Generative Models by Manifold Entropic Metricscited by 2
  11. 2025On the Identifiability of Latent Action Policiescited by 2
  12. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 2
  13. 2025Sequence Modeling with Spectral Mean Flowscited by 2
  14. 2025Equivariance by Contrast: Identifiable Equivariant Embeddings from Unlabeled Finite Group Actionscited by 2
  15. 2025Probabilistic Modelling is Sufficient for Causal Inferencecited by 2
  16. 2026AR-Flow VAE: A Structured Autoregressive Flow Prior Variational Autoencoder for Unsupervised Blind Source Separationcited by 2
  17. 2026StrADiff: A Structured Source-Wise Adaptive Diffusion Framework for Linear and Nonlinear Blind Source Separationcited by 2
  18. 2026A General Representation-Based Approach to Multi-Source Domain Adaptationcited by 2
  19. 2024Open Problem: Active Representation Learningcited by 1
  20. 2024Unsupervised discovery of the shared and private geometry in multi-view datacited by 1
  21. 2024Identifiable Representation and Model Learning for Latent Dynamic Systemscited by 1
  22. 2025Identifiable Autoregressive Variational Autoencoders for Nonlinear and Nonstationary Spatio-Temporal Blind Source Separationcited by 1
  23. 2026Extracting Governing Equations from Latent Dynamics via Multi-View Contrastive Learningcited by 1
  24. 2024Identifying latent state transition in non-linear dynamical systemscited by 0
  25. 2025Is Supervised Learning Really That Different from Unsupervised?cited by 0
  26. 2025Learning Treatment Representations for Downstream Instrumental Variable Regressioncited by 0
  27. 2025EEG-D3: A Solution to the Hidden Overfitting Problem of Deep Learning Modelscited by 0
  28. 2026From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flowscited by 0
  29. 2026A 1/R Law for Kurtosis Contrast in Balanced Mixturescited by 0
  30. 2026Reservoir Subspace Injection for Online ICA under Top-n Whiteningcited by 0
  31. 2026On the Invariance and Generality of Neural Scaling Lawscited by 0
  32. 2026Identifiable Multimodal Causal Representation Learning under Partial Latent Sharingcited by 0
  33. 2026Disentanglement Beyond Generative Models with Riemannian ICAcited by 0
  34. 2026Learning task-specific subspaces via interventional post-training of speech foundation modelscited by 0
  35. 2026Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiabilitycited by 0
  36. 2026AMICA-Python: Adaptive Mixture Independent Component Analysis with Anderson Accelerationcited by 0