mlsanityData to 7 Oct 2026

Improving VAE-based Representation Learning

arXiv 2205.14539 ↗First posted 28 May 2022stat.ML cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Latent variable models like the Variational Auto-Encoder (VAE) are commonly used to learn representations of images. However, for downstream tasks like semantic classification, the representations learned by VAE are less competitive than other non-latent variable models. This has led to some speculations that latent variable models may be fundamentally unsuitable for representation learning. In this work, we study what properties are required for good representations and how different VAE structure choices could affect the learned properties. We show that by using a decoder that prefers to learn local features, the remaining global features can be well captured by the latent, which significantly improves performance of a downstream classification task. We further apply the proposed model to semi-supervised learning tasks and demonstrate improvements in data efficiency.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 73 references that are in the map. First posted on arXiv 28 May 2022.

Builds on

45 of 73 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  11. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  12. 2016Understanding intermediate layers using linear classifier probescited by 1,279
33 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  2. 2016Pixel Recurrent Neural Networkscited by 814
  3. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  4. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  5. 2019Designing and Interpreting Probes with Control Taskscited by 608
  6. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  7. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  8. 2015Adversarial Autoencoderscited by 359
  9. 2016Adversarial Feature Learningcited by 350
  10. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  11. 2016LIFT: Learned Invariant Feature Transformcited by 305
  12. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  13. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  14. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  15. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
  16. 2018Inference Suboptimality in Variational Autoencoderscited by 131
  17. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  18. 2017Unsupervised Learning by Predicting Noisecited by 97
  19. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
  20. 2019Semi-Supervised Learning with Normalizing Flowscited by 76
  21. 2017Learning Disentangled Representations with Semi-Supervised Deep Generative Modelscited by 72
  22. 2016PixelVAE: A Latent Variable Model for Natural Imagescited by 69
  23. 2020Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Featurescited by 64
  24. 2019HiLLoC: Lossless Image Compression with Hierarchical Latent Variable Modelscited by 36
  25. 2020Learning Optimal Representations with the Decodable Information Bottleneckcited by 32
  26. 2021On the Out-of-distribution Generalization of Probabilistic Image Modellingcited by 32
  27. 2018Amortized Inference Regularizationcited by 30
  28. 2018Correction of AI systems by linear discriminants: Probabilistic foundationscited by 17
  29. 2022Generalization Gap in Amortized Inferencecited by 10
  30. 2019Invariant-equivariant representation learning for multi-class datacited by 6
  31. 2020High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionalitycited by 6
  32. 2022Parallel Neural Local Lossless Compressioncited by 4
  33. 2021Locally-Contextual Nonlinear CRFs for Sequence Labelingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 0; 2026: 1.

6 papers.

  1. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 19
  2. 2024A Probabilistic Model Behind Self-Supervised Learningcited by 5
  3. 2023Structural Adversarial Objectives for Self-Supervised Representation Learningcited by 2
  4. 2024Gaussian Mixture Vector Quantization with Aggregated Categorical Posteriorcited by 2
  5. 2026Learning Sparse Visual Representations via Spatial-Semantic Factorizationcited by 1
  6. 2023A Saliency-based Clustering Framework for Identifying Aberrant Predictionscited by 0