# Variational Laplace Autoencoders

arXiv: https://arxiv.org/abs/2211.17267 · HTML: https://mlsanity.com/paper/2211.17267/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks](https://mlsanity.com/paper/1701.04722/) (2017)
- [On the number of response regions of deep feed forward networks with piece-wise linear activations](https://mlsanity.com/paper/1312.6098/) (2013)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Semi-Amortized Variational Autoencoders](https://mlsanity.com/paper/1802.02550/) (2018)
- [Auxiliary Deep Generative Models](https://mlsanity.com/paper/1602.05473/) (2016)
- [PixelVAE: A Latent Variable Model for Natural Images](https://mlsanity.com/paper/1611.05013/) (2016)
- [Improving Variational Auto-Encoders using Householder Flow](https://mlsanity.com/paper/1611.09630/) (2016)
- [The Variational Gaussian Process](https://mlsanity.com/paper/1511.06499/) (2015)
- [On the challenges of learning with inference networks on sparse, high-dimensional data](https://mlsanity.com/paper/1710.06085/) (2017)

## Built on by (most cited first, 7 of 7)

- [Curvature-Sensitive Predictive Coding with Approximate Laplace Monte Carlo](https://mlsanity.com/paper/2303.04976/) (2023)
- [Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis](https://mlsanity.com/paper/2506.00849/) (2025)
- [An Efficient Difference-of-Convex Solver for Privacy Funnel](https://mlsanity.com/paper/2403.04778/) (2024)
- [Neural Network Characterization and Entropy Regulated Data Balancing through Principal Component Analysis](https://mlsanity.com/paper/2312.01392/) (2023)
- [Beyond Diagonal Covariance: Flexible Posterior VAEs via Free-Form Injective Flows](https://mlsanity.com/paper/2506.01522/) (2025)
- [Likelihood-Free Variational Autoencoders](https://mlsanity.com/paper/2504.17622/) (2025)
- [Metabolic cost of information processing in Poisson variational autoencoders](https://mlsanity.com/paper/2602.13421/) (2026)

## Abstract (arXiv)

Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. However, such amortized variational inference faces two challenges: (1) the limited posterior expressiveness of fully-factorized Gaussian assumption and (2) the amortization error of the inference model. We present a novel approach that addresses both challenges. First, we focus on ReLU networks with Gaussian output and illustrate their connection to probabilistic PCA. Building on this observation, we derive an iterative algorithm that finds the mode of the posterior and apply full-covariance Gaussian posterior approximation centered on the mode. Subsequently, we present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models. Based on the Laplace approximation of the latent variable posterior, VLAEs enhance the expressiveness of the posterior while reducing the amortization error. Empirical results on MNIST, Omniglot, Fashion-MNIST, SVHN and CIFAR10 show that the proposed approach significantly outperforms other recent amortized or iterative methods on the ReLU networks.