mlsanityData to 14 Oct 2026

The Neglected Sibling: Isotropic Gaussian Posterior for VAE

arXiv 2110.07383 ↗First posted 14 Oct 2021cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep generative models have been widely used in several areas of NLP, and various techniques have been proposed to augment them or address their training challenges. In this paper, we propose a simple modification to Variational Autoencoders (VAEs) by using an Isotropic Gaussian Posterior (IGP) that allows for better utilisation of their latent representation space. This model avoids the sub-optimal behavior of VAEs related to inactive dimensions in the representation space. We provide both theoretical analysis, and empirical evidence on various datasets and tasks that show IGP leads to consistent improvement on several quantitative and qualitative grounds, from downstream task performance and sample efficiency to robustness. Additionally, we give insights about the representational properties encouraged by IGP and also show that its gain generalises to image domain as well.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

24 of 34 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2021Convex optimizationcited by 2,515
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  6. 2015Generating Sentences from a Continuous Spacecited by 612
  7. 2015Importance Weighted Autoencoderscited by 456
  8. 2018Understanding disentangling in β-VAEcited by 335
  9. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  10. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  11. 2020Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Spacecited by 114
  12. 2015The Goldilocks Principle: Reading Children's Books with Explicit Memory Representationscited by 108
12 more
  1. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  2. 2017Improved Variational Autoencoders for Text Modeling using Dilated Convolutionscited by 81
  3. 2018Tighter Variational Bounds are Not Necessarily Bettercited by 76
  4. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  5. 2019A Surprisingly Effective Fix for Deep Latent Variable Modeling of Textcited by 32
  6. 2018StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsingcited by 22
  7. 2020Preventing Posterior Collapse with Levenshtein Variational Autoencodercited by 12
  8. 2019Effective Estimation of Deep Generative Language Modelscited by 9
  9. 2019On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generationcited by 9
  10. 2020Do sequence-to-sequence VAEs learn global features of sentences?cited by 6
  11. 2019A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matchingcited by 2
  12. 2021Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classificationcited by 1

Built on by

Not cited yet.