mlsanityData to 7 Oct 2026

Identifiability of deep generative models without auxiliary information

arXiv 2206.10044 ↗First posted 20 Jun 2022cs.LG cs.AI math.STcited by 70

Not on the map. Why?

Abstract (arXiv)

We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders that are commonly used in practice. Unlike existing work, our analysis does not require weak supervision, auxiliary information, or conditioning in the latent space. Specifically, we show that for a broad class of generative (i.e. unsupervised) models with universal approximation capabilities, the side information u is not necessary: We prove identifiability of the entire generative model where we do not observe u and only observe the data x. The models we consider match autoencoder architectures used in practice that leverage mixture priors in the latent space and ReLU/leaky-ReLU activations in the encoder, such as VaDE and MFC-VAE. Our main result is an identifiability hierarchy that significantly generalizes previous work and exposes how different assumptions lead to different "strengths" of identifiability, and includes certain "vanilla" VAEs with isotropic Gaussian priors as a special case. For example, our weakest result establishes (unsupervised) identifiability up to an affine transformation, and thus partially resolves an open problem regarding model identifiability raised in prior work. These theoretical results are augmented with experiments on both simulated and real data.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Builds on 43 of its 102 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

43 of 102 references.

References · 43

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  3. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  4. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2017VAE with a VampPriorcited by 247
  8. 2014Understanding image representations by measuring their equivariance and equivalencecited by 240
  9. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  10. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  11. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 219
  12. 2016Variational Deep Embedding: An Unsupervised and Generative Approach to Clusteringcited by 219
31 more
  1. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  2. 2022Weakly supervised causal representation learningcited by 140
  3. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  4. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  5. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  6. 2020On Linear Identifiability of Learned Representationscited by 103
  7. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  8. 2021Independent mechanism analysis, a new concept?cited by 96
  9. 2020Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN)cited by 91
  10. 2020Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximatorscited by 85
  11. 2021Similarity and Matching of Neural Network Representationscited by 77
  12. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76
  13. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  14. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  15. 2021Learning latent causal graphs via mixture oraclescited by 55
  16. 2019The Usual Suspects? Reassessing Blame for VAE Posterior Collapsecited by 52
  17. 2018Learning Latent Subspaces in Variational Autoencoderscited by 51
  18. 2022Universal approximation property of invertible neural networkscited by 30
  19. 2021Multi-Facet Clustering Variational Autoencoderscited by 28
  20. 2021I Don't Need u: Identifiable Non-Linear ICA Without Side Informationcited by 21
  21. 2021An Embedding of ReLU Networks and an Analysis of their Identifiabilitycited by 20
  22. 2018Identifiability of Nonparametric Mixture Models and Bayes Optimal Clusteringcited by 15
  23. 2020Failure Modes of Variational Autoencoders and Their Effects on Downstream Taskscited by 15
  24. 2018Learning Latent Superstructures in Variational Autoencoders for Deep Multidimensional Clusteringcited by 14
  25. 2019Measurement Dependence Inducing Latent Causal Modelscited by 14
  26. 2020Generalization Properties of Optimal Transport GANs with Latent Distribution Learningcited by 13
  27. 2019Disentangling to Cluster: Gaussian Mixture Variational Ladder Autoencoderscited by 9
  28. 2019Identifying through Flows for Recovering Latent Representationscited by 8
  29. 2020Consistent Estimation of Identifiable Nonparametric Mixture Models from Grouped Observationscited by 8
  30. 2013When are Overcomplete Topic Models Identifiable? Uniqueness of Tensor Tucker Decompositions with Structured Sparsitycited by 7
  31. 2014Warped Mixtures for Nonparametric Cluster Shapescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 22; 2024: 17; 2025: 18; 2026: 12.

70 papers.

Sort:

By area:

  1. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  2. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  3. 2024On the Origins of Linear Representations in Large Language Modelscited by 76
  4. 2023Multi-View Causal Representation Learning with Partial Observabilitycited by 72
  5. 2023Causal Component Analysiscited by 56
  6. 2023Disentanglement via Latent Quantizationcited by 40
  7. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 34
  8. 2023Generalizing Nonlinear ICA Beyond Structural Sparsitycited by 33
  9. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 30
  10. 2024Unifying Causal Representation Learning with the Invariance Principlecited by 30
  11. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
  12. 2025Thought Communication in Multiagent Collaborationcited by 28
38 more
  1. 2023Multi-Domain Causal Representation Learning via Weak Distributional Invariancescited by 23
  2. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 22
  3. 2023Identifying Representations for Intervention Extrapolationcited by 21
  4. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 19
  5. 2024Non-negative Contrastive Learningcited by 18
  6. 2024Learning Discrete Concepts in Latent Hierarchical Modelscited by 17
  7. 2023Manifold Learning by Mixture Models of VAEs for Inverse Problemscited by 16
  8. 2023Generative Causal Representation Learning for Out-of-Distribution Motion Forecastingcited by 12
  9. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  10. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  11. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  12. 2023Emerging Synergies in Causality and Deep Generative Models: A Surveycited by 10
  13. 2024Distributional Principal Autoencoderscited by 10
  14. 2023On the Identifiability of Switching Dynamical Systemscited by 9
  15. 2024Identifiable Object-Centric Representation Learning via Probabilistic Slot Attentioncited by 9
  16. 2024Language Agents Meet Causality -- Bridging LLMs and Causal World Modelscited by 8
  17. 2023Learning Unknown Intervention Targets in Structural Causal Models from Heterogeneous Datacited by 7
  18. 2024Identifiability Guarantees for Causal Disentanglement from Purely Observational Datacited by 7
  19. 2025Robustness of Nonlinear Representation Learningcited by 7
  20. 2024Separating common from salient patterns with Contrastive Representation Learningcited by 6
  21. 2024Causal Representation Learning in Temporal Data via Single-Parent Decodingcited by 5
  22. 2023Neuro-Causal Factor Analysiscited by 4
  23. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectivescited by 4
  24. 2023When Graph Neural Network Meets Causality: Opportunities, Methodologies and An Outlookcited by 4
  25. 2025Transfer Learning in Latent Contextual Bandits with Covariate Shift Through Causal Transportabilitycited by 4
  26. 2025Near-Optimal Experiment Design in Linear non-Gaussian Cyclic Modelscited by 4
  27. 2025Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiabilitycited by 4
  28. 2023On the Identifiability of Quantized Factorscited by 3
  29. 2023A Causal Ordering Prior for Unsupervised Representation Learningcited by 3
  30. 2025Identifying Metric Structures of Deep Latent Variable Modelscited by 3
  31. 2025Causality-Inspired Robustness for Nonlinear Models via Representation Learningcited by 3
  32. 2025Mechanistic Independence: A Principle for Identifiable Disentangled Representationscited by 3
  33. 2025Modality Alignment across Trees on Heterogeneous Hyperbolic Manifoldscited by 3
  34. 2022Generalized Identifiability Bounds for Mixture Models with Grouped Samplescited by 2
  35. 2023Causally Disentangled Generative Variational AutoEncodercited by 2
  36. 2025Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layerscited by 2
  37. 2026StrADiff: A Structured Source-Wise Adaptive Diffusion Framework for Linear and Nonlinear Blind Source Separationcited by 2
  38. 2023Leveraging Task Structures for Improved Identifiability in Neural Network Representationscited by 1

50 most cited of 70 All 70 (JSON)