mlsanityData to 7 Oct 2026

Exploring the Latent Space of Autoencoders with Interventional Assays

arXiv 2106.16091 ↗First posted 30 Jun 2021cs.LG cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Autoencoders exhibit impressive abilities to embed the data manifold into a low-dimensional latent space, making them a staple of representation learning methods. However, without explicit supervision, which is often unavailable, the representation is usually uninterpretable, making analysis and principled progress challenging. We propose a framework, called latent responses, which exploits the locally contractive behavior exhibited by variational autoencoders to explore the learned manifold. More specifically, we develop tools to probe the representation using interventions in the latent space to quantify the relationships between latent variables. We extend the notion of disentanglement to take the learned generative process into account and consequently avoid the limitations of existing metrics that may rely on spurious correlations. Our analyses underscore the importance of studying the causal structure of the representation to improve performance on downstream tasks such as generation, interpolation, and inference of the factors of variation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 47 of its 78 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

47 of 78 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  5. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  6. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  7. 2019An Introduction to Variational Autoencoderscited by 997
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  11. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  12. 2018Understanding disentangling in β-VAEcited by 332
35 more
  1. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  2. 2017VAE with a VampPriorcited by 247
  3. 2019Learning Shape Templates with Structured Implicit Functionscited by 242
  4. 2015The Variational Fair Autoencodercited by 223
  5. 2019Diagnosing and Enhancing VAE Modelscited by 192
  6. 2018Hyperspherical Variational Auto-Encoderscited by 188
  7. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  8. 2019On the Fairness of Disentangled Representationscited by 144
  9. 2020Dynamical Variational Autoencoders: A Comprehensive Reviewcited by 127
  10. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  11. 2019Towards Visually Explaining Variational Autoencoderscited by 113
  12. 2019Are Disentangled Representations Helpful for Abstract Visual Reasoning?cited by 100
  13. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  14. 2018Taming VAEscited by 85
  15. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
  16. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  17. 2018Explorations in Homeomorphic Variational Auto-Encodingcited by 65
  18. 2020On the Transfer of Disentangled Representations in Realistic Settingscited by 63
  19. 2017Metrics for Deep Generative Modelscited by 53
  20. 2020Variational Autoencoder with Learned Latent Structurecited by 41
  21. 2020Variational Autoencoders with Riemannian Brownian Motion Priorscited by 34
  22. 2020Learning Flat Latent Manifolds with VAEscited by 33
  23. 2018Understanding Autoencoders with Information Theoretic Conceptscited by 29
  24. 2014Metrics for Probabilistic Geometriescited by 28
  25. 2015A Hierarchical Neural Autoencoder for Paragraphs and Documentscited by 26
  26. 2019Fast and Robust Shortest Paths on Manifolds Learned from Datacited by 25
  27. 2020Autoencoding Variational Autoencodercited by 24
  28. 2020Evaluating representations by the complexity of learning low-loss predictorscited by 16
  29. 2019Perceptual Generative Autoencoderscited by 15
  30. 2018Geodesic Clustering in Deep Generative Modelscited by 14
  31. 2018Deep Generative Models in the Real-World: An Open Challenge from Medical Imagingcited by 12
  32. 2020Geometry-Aware Hamiltonian Variational Auto-Encodercited by 10
  33. 2019Independent Subspace Analysis for Unsupervised Learning of Disentangled Representationscited by 8
  34. 2021On Linear Interpolation in the Latent Space of Deep Generative Modelscited by 8
  35. 2021Spatial Dependency Networks: Neural Layers for Improved Generative Image Modelingcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 4; 2024: 3; 2025: 2.

10 papers.

  1. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  2. 2023From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modelingcited by 21
  3. 2024Mapping the Multiverse of Latent Representationscited by 5
  4. 2023Terrain Diffusion Network: Climatic-Aware Terrain Generation with Geological Sketch Guidancecited by 3
  5. 2023Are We Using Autoencoders in a Wrong Way?cited by 2
  6. 2024Latent Space Characterization of Autoencoder Variantscited by 2
  7. 2025Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Modelscited by 2
  8. 2024Understanding Variational Autoencoders with Intrinsic Dimension and Information Imbalancecited by 1
  9. 2022Attribute Graphs Underlying Molecular Generative Models: Path to Learning with Limited Datacited by 0
  10. 2025From Points to Spheres: A Geometric Reinterpretation of Variational Autoencoderscited by 0