# Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA

arXiv: https://arxiv.org/abs/2106.09620 · HTML: https://mlsanity.com/paper/2106.09620/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning](https://mlsanity.com/paper/1805.08651/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA](https://mlsanity.com/paper/2002.11537/) (2020)
- [Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series](https://mlsanity.com/paper/2006.12107/) (2020)
- [Frequentist Consistency of Variational Bayes](https://mlsanity.com/paper/1705.03439/) (2017)
- [Causal Discovery with General Non-Linear Relationships Using Non-Linear ICA](https://mlsanity.com/paper/1904.09096/) (2019)
- [Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE](https://mlsanity.com/paper/2011.04798/) (2020)
- [Independent Innovation Analysis for Nonlinear Vector Autoregressive Process](https://mlsanity.com/paper/2006.10944/) (2020)
- [Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Method](https://mlsanity.com/paper/2001.01894/) (2020)
- [Relative gradient optimization of the Jacobian term in unsupervised deep learning](https://mlsanity.com/paper/2006.15090/) (2020)
- [Nonlinear Independent Component Analysis for Discrete-Time and Continuous-Time Signals](https://mlsanity.com/paper/2102.02876/) (2021)

## Built on by (most cited first, 30 of 64)

- [Weakly supervised causal representation learning](https://mlsanity.com/paper/2203.16437/) (2022)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [On the Identifiability of Nonlinear ICA: Sparsity and Beyond](https://mlsanity.com/paper/2206.07751/) (2022)
- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Causal Representation Learning from Multiple Distributions: A General Setting](https://mlsanity.com/paper/2402.05052/) (2024)
- [Temporally Disentangled Representation Learning](https://mlsanity.com/paper/2210.13647/) (2022)
- [Identifiability of latent-variable and structural-equation models: from linear to nonlinear](https://mlsanity.com/paper/2302.02672/) (2023)
- [Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation](https://mlsanity.com/paper/2307.02598/) (2023)
- [Causal Component Analysis](https://mlsanity.com/paper/2305.17225/) (2023)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [Provably Learning Object-Centric Representations](https://mlsanity.com/paper/2305.14229/) (2023)
- [Provable Compositional Generalization for Object-Centric Learning](https://mlsanity.com/paper/2310.05327/) (2023)
- [Subspace Identification for Multi-Source Domain Adaptation](https://mlsanity.com/paper/2310.04723/) (2023)
- [Learning nonparametric latent causal graphs with unknown interventions](https://mlsanity.com/paper/2306.02899/) (2023)
- [Generalizing Nonlinear ICA Beyond Structural Sparsity](https://mlsanity.com/paper/2311.00866/) (2023)
- [Indeterminacy in Generative Models: Characterization and Strong Identifiability](https://mlsanity.com/paper/2206.00801/) (2022)
- [Causal Representation Learning Made Identifiable by Grouping of Observational Variables](https://mlsanity.com/paper/2310.15709/) (2023)
- [Cross-Entropy Is All You Need To Invert the Data Generating Process](https://mlsanity.com/paper/2410.21869/) (2024)
- [Thought Communication in Multiagent Collaboration](https://mlsanity.com/paper/2510.20733/) (2025)
- [Conditional Mutual Information for Disentangled Representations in Reinforcement Learning](https://mlsanity.com/paper/2305.14133/) (2023)
- [CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Process](https://mlsanity.com/paper/2401.14535/) (2024)
- [Temporally Disentangled Representation Learning under Unknown Nonstationarity](https://mlsanity.com/paper/2310.18615/) (2023)
- [Identifying Representations for Intervention Extrapolation](https://mlsanity.com/paper/2310.04295/) (2023)
- [On the Identification of Temporally Causal Representation with Instantaneous Dependence](https://mlsanity.com/paper/2405.15325/) (2024)
- [Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learning](https://mlsanity.com/paper/2406.14302/) (2024)
- [Learning Latent Causal Dynamics](https://mlsanity.com/paper/2202.04828/) (2022)
- [Causal Temporal Representation Learning with Nonstationary Sparse Transition](https://mlsanity.com/paper/2409.03142/) (2024)
- [Towards Interpretable Deep Generative Models via Causal Representation Learning](https://mlsanity.com/paper/2504.11609/) (2025)

## Abstract (arXiv)

We introduce a new general identifiable framework for principled disentanglement referred to as Structured Nonlinear Independent Component Analysis (SNICA). Our contribution is to extend the identifiability theory of deep generative models for a very broad class of structured models. While previous works have shown identifiability for specific classes of time-series models, our theorems extend this to more general temporal structures as well as to models with more complex structures such as spatial dependencies. In particular, we establish the major result that identifiability for this framework holds even in the presence of noise of unknown distribution. Finally, as an example of our framework's flexibility, we introduce the first nonlinear ICA model for time-series that combines the following very useful properties: it accounts for both nonstationarity and autocorrelation in a fully unsupervised setting; performs dimensionality reduction; models hidden states; and enables principled estimation and inference by variational maximum-likelihood.