mlsanityData to 7 Oct 2026

Indeterminacy in Generative Models: Characterization and Strong Identifiability

arXiv 2206.00801 ↗First posted 2 Jun 2022stat.ML cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Most modern probabilistic generative models, such as the variational autoencoder (VAE), have certain indeterminacies that are unresolvable even with an infinite amount of data. Different tasks tolerate different indeterminacies, however recent applications have indicated the need for strongly identifiable models, in which an observation corresponds to a unique latent code. Progress has been made towards reducing model indeterminacies while maintaining flexibility, and recent work excludes many--but not all--indeterminacies. In this work, we motivate model-identifiability in terms of task-identifiability, then construct a theoretical framework for analyzing the indeterminacies of latent variable models, which enables their precise characterization in terms of the generator function and prior distribution spaces. We reveal that strong identifiability is possible even with highly flexible nonlinear generators, and give two such examples. One is a straightforward modification of iVAE (arXiv:1907.04809 [stat.ML]); the other uses triangular monotonic maps, leading to novel connections between optimal transport and identifiability.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

25 of 56 references.

References · 25

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  3. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  4. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  5. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 278
  6. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 270
  7. 2020Weakly-Supervised Disentanglement Without Compromisescited by 237
  8. 2018Neural Autoregressive Flowscited by 207
  9. 2021Towards Causal Representation Learningcited by 207
  10. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  11. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 158
  12. 2022Weakly supervised causal representation learningcited by 140
13 more
  1. 2020Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Codingcited by 121
  2. 2021Independent mechanism analysis, a new concept?cited by 96
  3. 2019Unconstrained Monotonic Neural Networkscited by 91
  4. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  5. 2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
  6. 2020Weakly Supervised Disentangled Generative Causal Representation Learningcited by 67
  7. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  8. 2019Sum-of-Squares Polynomial Flowcited by 62
  9. 2019Causal Discovery with General Non-Linear Relationships Using Non-Linear ICAcited by 58
  10. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  11. 2021Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learningcited by 30
  12. 2021Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Ratescited by 16
  13. 2019Independent Subspace Analysis for Unsupervised Learning of Disentangled Representationscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 13; 2024: 8; 2025: 8; 2026: 4.

33 papers.

Sort:

By area:

  1. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 105
  2. 2023Causal Component Analysiscited by 56
  3. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  4. 2023Causal normalizing flows: from theory to practicecited by 49
  5. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  6. 2023Disentanglement via Latent Quantizationcited by 40
  7. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 21
  8. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  9. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  10. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 12
  11. 2025Identifiability of Deep Polynomial Neural Networkscited by 12
  12. 2025DeCaFlow: A deconfounding causal generative modelcited by 8
21 more
  1. 2023Independent Mechanism Analysis and the Manifold Hypothesiscited by 7
  2. 2025Robustness of Nonlinear Representation Learningcited by 7
  3. 2024Propensity Score Alignment of Unpaired Multimodal Datacited by 6
  4. 2024On the Identifiability of Sparse ICA without Assuming Non-Gaussianitycited by 5
  5. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectivescited by 4
  6. 2023Consistent End-to-End Estimation for Counterfactual Fairnesscited by 4
  7. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  8. 2023Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distributioncited by 3
  9. 2023Estimating treatment effects from single-arm trials via latent-variable modelingcited by 1
  10. 2024Causal Order Discovery based on Monotonic SCMscited by 1
  11. 2026Nonlinear multi-study sparse factor analysiscited by 1
  12. 2026A Unifying Framework for Unsupervised Concept Extractioncited by 1
  13. 2024Identifying latent state transition in non-linear dynamical systemscited by 0
  14. 2024Unsupervised Object Detection with Theoretical Guaranteescited by 0
  15. 2024Celcomen: spatial causal disentanglement for single-cell and tissue perturbation modelingcited by 0
  16. 2024Causally Consistent Normalizing Flowcited by 0
  17. 2025On Measuring Intrinsic Causal Attributions in Deep Neural Networkscited by 0
  18. 2025Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systemscited by 0
  19. 2025Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimalitycited by 0
  20. 2026Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applicationscited by 0
  21. 2026Beyond ICA: Identifiability by Symmetry Breakingcited by 0