# Theoretical Convergence Guarantees for Variational Autoencoders

arXiv: https://arxiv.org/abs/2410.16750 · HTML: https://mlsanity.com/paper/2410.16750/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2024.

## Built on by (most cited first, 4 of 4)

- [Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference](https://mlsanity.com/paper/2505.21721/) (2025)
- [Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models](https://mlsanity.com/paper/2607.04775/) (2026)
- [PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders](https://mlsanity.com/paper/2609.05212/) (2026)
- [Latent Guided Sampling for Combinatorial Optimization](https://mlsanity.com/paper/2506.03672/) (2025)

## Abstract (arXiv)

Variational Autoencoders (VAE) are popular generative models used to sample from complex data distributions. Despite their empirical success in various machine learning tasks, significant gaps remain in understanding their theoretical properties, particularly regarding convergence guarantees. This paper aims to bridge that gap by providing non-asymptotic convergence guarantees for VAE trained using both Stochastic Gradient Descent and Adam algorithms. We derive a convergence rate of O(log n / √n), where n is the number of iterations of the optimization algorithm, with explicit dependencies on the batch size, the number of variational samples, and other key hyperparameters. Our theoretical analysis applies to both Linear VAE and Deep Gaussian VAE, as well as several VAE variants, including β-VAE and IWAE. Additionally, we empirically illustrate the impact of hyperparameters on convergence, offering new insights into the theoretical understanding of VAE training.