mlsanityData to 7 Oct 2026

Variational Lossy Autoencoder

arXiv 1611.02731 ↗First posted 8 Nov 2016cs.LG stat.MLcited by 175

Not on the map. Why?

Abstract (arXiv)

Representation learning seeks to expose certain aspects of observed data in a learned representation that's amenable to downstream tasks like classification. For instance, a good representation for 2D images might be one that describes only global structure and discards information about detailed texture. In this paper, we present a simple but principled method to learn such global representations by combining Variational Autoencoder (VAE) with neural autoregressive models such as RNN, MADE and PixelRNN/CNN. Our proposed VAE model allows us to have control over what the global latent code can learn and , by designing the architecture accordingly, we can force the global latent code to discard irrelevant information such as texture in 2D images, and hence the VAE only "autoencodes" data in a lossy fashion. In addition, by leveraging autoregressive models as both prior distribution p(z) and decoding distribution p(x|z), we can greatly improve generative modeling performance of VAEs, achieving new state-of-the-art results on MNIST, OMNIGLOT and Caltech-101 Silhouettes density estimation tasks.

Key facts. Cited by 175 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 54; 2022: 41; 2023: 25; 2024: 25; 2025: 14; 2026: 16.

175 papers.

Sort:

By area:

  1. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  2. 2021Score-based Generative Modeling in Latent Spacecited by 634
  3. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  4. 2022From data to functa: Your data point is a function and you can treat it like onecited by 183
  5. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
  6. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  7. 2023Learning to Act without Actionscited by 116
  8. 2021Mixed supervision for surface-defect detection: from weakly to fully supervised learningcited by 101
  9. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  10. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  11. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  12. 2021Variational Transformer Networks for Layout Generationcited by 88
38 more
  1. 2023GIVT: Generative Infinite-Vocabulary Transformerscited by 80
  2. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Priorcited by 78
  3. 2022Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Featurescited by 63
  4. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  5. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  6. 2021Lossy Compression for Lossless Predictioncited by 51
  7. 2022A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inferencecited by 50
  8. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  9. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  10. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  11. 2024Autoregressive Models in Vision: A Surveycited by 39
  12. 2022Snowflake Point Deconvolution for Point Cloud Completion and Generation with Skip-Transformercited by 38
  13. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  14. 2023Autoregressive Conditional Neural Processescited by 32
  15. 2022VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programmingcited by 31
  16. 2021Personalized Trajectory Prediction via Distribution Discriminationcited by 30
  17. 2021DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendationcited by 28
  18. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  19. 2021Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencodercited by 27
  20. 2021Interpretable Unsupervised Diversity Denoising and Artefact Removalcited by 26
  21. 2022Efficient-VDVAE: Less is morecited by 26
  22. 2022Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Casecited by 26
  23. 2021Diffusion Priors In Variational Autoencoderscited by 25
  24. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  25. 2021Variable-rate discrete representation learningcited by 24
  26. 2022A Geometric Perspective on Variational Autoencoderscited by 24
  27. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  28. 2025Generative Latent Coding for Ultra-Low Bitrate Image and Video Compressioncited by 22
  29. 2021Fairness without the sensitive attribute via Causal Variational Autoencodercited by 21
  30. 2025Generative Latent Coding for Ultra-Low Bitrate Image Compressioncited by 21
  31. 2024How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces?cited by 19
  32. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  33. 2021UAV Localization Using Autoencoded Satellite Imagescited by 17
  34. 2021Go with the Flows: Mixtures of Normalizing Flows for Point Cloud Generation and Reconstructioncited by 16
  35. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  36. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  37. 2021Behavior-Driven Synthesis of Human Dynamicscited by 14
  38. 2024DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wildcited by 14

50 most cited of 175 All 175 (JSON)