mlsanityData to 7 Oct 2026

Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

arXiv 2106.09620 ↗First posted 17 Jun 2021stat.ML cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

We introduce a new general identifiable framework for principled disentanglement referred to as Structured Nonlinear Independent Component Analysis (SNICA). Our contribution is to extend the identifiability theory of deep generative models for a very broad class of structured models. While previous works have shown identifiability for specific classes of time-series models, our theorems extend this to more general temporal structures as well as to models with more complex structures such as spatial dependencies. In particular, we establish the major result that identifiability for this framework holds even in the presence of noise of unknown distribution. Finally, as an example of our framework's flexibility, we introduce the first nonlinear ICA model for time-series that combines the following very useful properties: it accounts for both nonstationarity and autocorrelation in a fully unsupervised setting; performs dimensionality reduction; models hidden states; and enables principled estimation and inference by variational maximum-likelihood.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 19 of its 46 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

19 of 46 references.

References · 19

  1. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  2. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  3. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  4. 2017Look, Listen and Learncited by 452
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  8. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  9. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  10. 2020Uncovering the structure of clinical EEG signals with self-supervised learningcited by 109
  11. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  12. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76
7 more
  1. 2017Frequentist Consistency of Variational Bayescited by 60
  2. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  3. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  4. 2020Independent Innovation Analysis for Nonlinear Vector Autoregressive Processcited by 18
  5. 2020Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Methodcited by 16
  6. 2020Relative gradient optimization of the Jacobian term in unsupervised deep learningcited by 16
  7. 2021Nonlinear Independent Component Analysis for Discrete-Time and Continuous-Time Signalscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 22; 2024: 11; 2025: 10; 2026: 9.

64 papers.

Sort:

By area:

  1. 2022Weakly supervised causal representation learningcited by 140
  2. 2022Learnable latent embeddings for joint behavioral and neural analysiscited by 113
  3. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  4. 2022On the Identifiability of Nonlinear ICA: Sparsity and Beyondcited by 84
  5. 2022Identifiability of deep generative models without auxiliary informationcited by 70
  6. 2024Causal Representation Learning from Multiple Distributions: A General Settingcited by 67
  7. 2022Temporally Disentangled Representation Learningcited by 59
  8. 2023Identifiability of latent-variable and structural-equation models: from linear to nonlinearcited by 58
  9. 2023Causal Component Analysiscited by 56
  10. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  11. 2021Identifiable Deep Generative Models via Sparse Decodingcited by 54
  12. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
38 more
  1. 2023Provably Learning Object-Centric Representationscited by 45
  2. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  3. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
  4. 2023Learning nonparametric latent causal graphs with unknown interventionscited by 34
  5. 2022Indeterminacy in Generative Models: Characterization and Strong Identifiabilitycited by 33
  6. 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 33
  7. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 30
  8. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 30
  9. 2025Thought Communication in Multiagent Collaborationcited by 28
  10. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  11. 2023Identifying Representations for Intervention Extrapolationcited by 21
  12. 2023Temporally Disentangled Representation Learning under Unknown Nonstationaritycited by 21
  13. 2024CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Processcited by 20
  14. 2024Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learningcited by 16
  15. 2024On the Identification of Temporally Causal Representation with Instantaneous Dependencecited by 15
  16. 2022Learning Latent Causal Dynamicscited by 13
  17. 2022An Empirical Study on Disentanglement of Negative-free Contrastive Learningcited by 12
  18. 2024Causal Temporal Representation Learning with Nonstationary Sparse Transitioncited by 12
  19. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  20. 2023Identifying Semantic Component for Robust Molecular Property Predictioncited by 10
  21. 2024Counterfactual Generative Modeling with Variational Causal Inferencecited by 10
  22. 2023On the Identifiability of Switching Dynamical Systemscited by 9
  23. 2023An Interventional Perspective on Identifiability in Gaussian LTI Systems with Independent Component Analysiscited by 9
  24. 2024Self-supervised contrastive learning performs non-linear system identificationcited by 9
  25. 2025Towards Identifiability of Hierarchical Temporal Causal Representation Learningcited by 7
  26. 2025Learning General Causal Structures with Hidden Dynamic Process for Climate Analysiscited by 6
  27. 2023Identifiable Feature Learning for Spatial Data with Nonlinear ICAcited by 4
  28. 2025Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learningcited by 4
  29. 2025Efficient Deconvolution in Populational Inverse Problemscited by 4
  30. 2025Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learningcited by 4
  31. 2026Causal Representation Learning from General Environments under Nonparametric Mixingcited by 4
  32. 2023On the Identifiability of Quantized Factorscited by 3
  33. 2023Conditionally Invariant Representation Learning for Disentangling Cellular Heterogeneitycited by 3
  34. 2022On Pitfalls of Identifiability in Unsupervised Learning. A Note on: "Desiderata for Representation Learning: A Causal Perspective"cited by 2
  35. 2024Tree-based variational inference for Poisson log-normal modelscited by 2
  36. 2025Structured Kernel Regression VAE: A Computationally Efficient Surrogate for GP-VAEs in ICAcited by 2
  37. 2023Discovering Predictable Latent Factors for Time Series Forecastingcited by 1
  38. 2023Leveraging Task Structures for Improved Identifiability in Neural Network Representationscited by 1

50 most cited of 64 All 64 (JSON)