mlsanityData to 14 Oct 2026

Identifiable Deep Generative Models via Sparse Decoding

arXiv 2110.10804 ↗First posted 20 Oct 2021stat.ML cs.LG stat.MEcited by 54

Not on the map. Why?

Abstract (arXiv)

We develop the sparse VAE for unsupervised representation learning on high-dimensional data. The sparse VAE learns a set of latent factors (representations) which summarize the associations in the observed data features. The underlying model is sparse in that each observed feature (i.e. each dimension of the data) depends on a small subset of the latent factors. As examples, in ratings data each movie is only described by a few genres; in text data each word is only applicable to a few topics; in genomics, each gene is active in only a few biological processes. We prove such sparse deep generative models are identifiable: with infinite data, the true model parameters can be learned. (In contrast, most deep generative models are not identifiable.) We empirically study the sparse VAE with both simulated and real data. We find that it recovers meaningful latent factors and has smaller heldout reconstruction error than related methods.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 19 of its 53 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

19 of 53 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  3. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  4. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  5. 2018Variational Autoencoders for Collaborative Filteringcited by 310
  6. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
  7. 2021Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICAcited by 160
  8. 2019On the Fairness of Disentangled Representationscited by 144
  9. 2018A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applicationscited by 136
  10. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  11. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  12. 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
7 more
  1. 2017Model Selection in Bayesian Neural Networks via Horseshoe Priorscited by 52
  2. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  3. 2014Bayesian linear regression with sparse priorscited by 40
  4. 2017Adaptive Estimation in Structured Factor Models with Applications to Overlapping Clusteringcited by 19
  5. 2020Vintage Factor Analysis with Varimax Performs Statistical Inferencecited by 17
  6. 2021Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularizationcited by 17
  7. 2021Bayesian Pyramids: Identifiable Multilayer Discrete Latent Structure Models for Discrete Datacited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 15; 2024: 12; 2025: 14; 2026: 7.

54 papers.

Sort:

By area:

  1. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  2. 2023Nonparametric Identifiability of Causal Representations from Unknown Interventionscited by 106
  3. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  4. 2022Linear Causal Disentanglement via Interventionscited by 84
  5. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 58
  6. 2022Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learningcited by 44
  7. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  8. 2023Disentanglement via Latent Quantizationcited by 40
  9. 2024A Sparsity Principle for Partially Observable Causal Representation Learningcited by 35
  10. 2022Learning Causal Representations of Single Cells via Sparse Mechanism Shift Modelingcited by 34
  11. 2023Causal Representation Learning Made Identifiable by Grouping of Observational Variablescited by 32
  12. 2024Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Modelscited by 29
38 more
  1. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 23
  2. 2023Identifying Representations for Intervention Extrapolationcited by 21
  3. 2023Object-centric architectures enable efficient causal representation learningcited by 21
  4. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 20
  5. 2024The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applicationscited by 18
  6. 2022When are Post-hoc Conceptual Explanations Identifiable?cited by 14
  7. 2024All or None: Identifiable Linear Properties of Next-token Predictors in Language Modelingcited by 14
  8. 2024Tripod: Three Complementary Inductive Biases for Disentangled Representation Learningcited by 13
  9. 2024Causal Representation Learning from Multimodal Biomedical Observationscited by 13
  10. 2025Towards Interpretable Deep Generative Models via Causal Representation Learningcited by 13
  11. 2025From Isolation to Entanglement: When Do Interpretability Methods Identify and Disentangle Known Concepts?cited by 7
  12. 2022Expressive architectures enhance interpretability of dynamics-based neural population modelscited by 6
  13. 2024On the Identifiability of Sparse ICA without Assuming Non-Gaussianitycited by 5
  14. 2023Neuro-Causal Factor Analysiscited by 4
  15. 2023Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectivescited by 4
  16. 2025Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learningcited by 4
  17. 2025When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspectivecited by 4
  18. 2025Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiabilitycited by 4
  19. 2022Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAEcited by 3
  20. 2023On the Identifiability of Quantized Factorscited by 3
  21. 2023Conditionally Invariant Representation Learning for Disentangling Cellular Heterogeneitycited by 3
  22. 2023Bayesian Transfer Learningcited by 3
  23. 2025Causality-Inspired Robustness for Nonlinear Models via Representation Learningcited by 3
  24. 2025Mechanistic Independence: A Principle for Identifiable Disentangled Representationscited by 3
  25. 2025Spatial Deconfounder: Interference-Aware Deconfounding for Spatial Causal Inferencecited by 3
  26. 2025Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layerscited by 2
  27. 2025Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layercited by 2
  28. 2026Content-Style Identification via Differential Independencecited by 2
  29. 2024Linear causal disentanglement via higher-order cumulantscited by 1
  30. 2024Discovering Latent Causal Graphs from Spatiotemporal Datacited by 1
  31. 2025Intervening to Learn and Compose Causally Disentangled Representationscited by 1
  32. 2026Nonlinear multi-study sparse factor analysiscited by 1
  33. 2026Beyond identifiability: Learning causal representations with few environments and finite samplescited by 1
  34. 2026Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixingcited by 1
  35. 2026Unsupervised Causal Abstractions Discoverycited by 1
  36. 2023SPARLING: Learning Latent Representations with Extremely Sparse Activationscited by 0
  37. 2024LASERS: LAtent Space Encoding for Representations with Sparsity for Generative Modelingcited by 0
  38. 2025Time-varying Factor Augmented Vector Autoregression with Grouped Sparse Autoencodercited by 0

50 most cited of 54 All 54 (JSON)