mlsanityData to 7 Oct 2026

Semi-Amortized Variational Autoencoders

arXiv 1802.02550 ↗First posted 7 Feb 2018stat.ML cs.CL cs.LGcited by 83

Not on the map. Why?

Abstract (arXiv)

Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameters. We propose a hybrid approach, to use AVI to initialize the variational parameters and run stochastic variational inference (SVI) to refine them. Crucially, the local SVI procedure is itself differentiable, so the inference network and generative model can be trained end-to-end with gradient-based optimization. This semi-amortized approach enables the use of rich generative models without experiencing the posterior-collapse phenomenon common in training VAEs for problems like text generation. Experiments show this approach outperforms strong autoregressive and variational baselines on standard text and image datasets.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 20; 2023: 15; 2024: 7; 2025: 7; 2026: 11.

83 papers.

Sort:

By area:

  1. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  2. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  3. 2021COIN: COmpression with Implicit Neural representationscited by 232
  4. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  5. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  6. 2022An Introduction to Neural Data Compressioncited by 78
  7. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  8. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  9. 2023Conditional score-based diffusion models for Bayesian inference in infinite dimensionscited by 37
  10. 2024Neural Methods for Amortized Inferencecited by 36
  11. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  12. 2023Amortized Variational Inference: When and Why?cited by 28
38 more
  1. 2022Hybrid Predictive Coding: Inferring, Fast and Slowcited by 25
  2. 2022Reliable amortized variational inference with physics-based latent distribution correctioncited by 25
  3. 2021Preconditioned training of normalizing flows for variational inference in inverse problemscited by 19
  4. 2022Flexible Neural Image Compression via Code Editingcited by 19
  5. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  6. 2021Certifiably Robust Variational Autoencoderscited by 14
  7. 2021Finetuning Pretrained Transformers into Variational Autoencoderscited by 14
  8. 2025Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modelingcited by 14
  9. 2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
  10. 2022Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priorscited by 13
  11. 2021VAE Approximation Error: ELBO and Exponential Familiescited by 12
  12. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  13. 2024Poisson Variational Autoencodercited by 11
  14. 2021Latent Space Refinement for Deep Generative Modelscited by 10
  15. 2021Implicit Riemannian Concave Potential Mapscited by 10
  16. 2021Applications of the Free Energy Principle to Machine Learning and Neurosciencecited by 9
  17. 2022Bit Allocation using Optimizationcited by 9
  18. 2023Deep Backtracking Counterfactuals for Causally Compliant Explanationscited by 8
  19. 2022Variational Laplace Autoencoderscited by 7
  20. 2023Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Networkcited by 7
  21. 2021Reducing the Amortization Gap in Variational Autoencoders: A Bayesian Random Function Approachcited by 6
  22. 2021Regularizing Variational Autoencoder with Diversity and Uncertainty Awarenesscited by 6
  23. 2022Forget-me-not! Contrastive Critics for Mitigating Posterior Collapsecited by 6
  24. 2022Optimization for Amortized Inverse Problemscited by 6
  25. 2021Generative Text Modeling through Short Run Inferencecited by 5
  26. 2022Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrievalcited by 5
  27. 2021Unsupervised Representation Learning via Neural Activation Codingcited by 4
  28. 2022DGNet: Distribution Guided Efficient Learning for Oil Spill Image Segmentationcited by 4
  29. 2023CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapsecited by 4
  30. 2024CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inferencecited by 4
  31. 2024Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equationscited by 4
  32. 2024Latent Diffusion Models for Controllable RNA Sequence Generationcited by 4
  33. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processescited by 4
  34. 2022Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAEcited by 3
  35. 2022Learning to Drop Out: An Adversarial Approach to Training Sequence VAEscited by 3
  36. 2024Explicit and data-Efficient Encoding via Gradient Flowcited by 3
  37. 2021Improving Actor-Critic Reinforcement Learning via Hamiltonian Monte Carlo Methodcited by 2
  38. 2021Enhancing audio quality for expressive Neural Text-to-Speechcited by 2

50 most cited of 83 All 83 (JSON)