mlsanityData to 14 Oct 2026

On PAC-Bayesian reconstruction guarantees for VAEs

arXiv 2202.11455 ↗First posted 23 Feb 2022cs.LG cs.CV math.STcited by 20

Not on the map. Why?

Abstract (arXiv)

Despite its wide use and empirical successes, the theoretical understanding and study of the behaviour and performance of the variational autoencoder (VAE) have only emerged in the past few years. We contribute to this recent line of work by analysing the VAE's reconstruction ability for unseen test data, leveraging arguments from the PAC-Bayes theory. We provide generalisation bounds on the theoretical reconstruction error, and provide insights on the regularisation effect of VAE objectives. We illustrate our theoretical results with supporting experiments on classical benchmark datasets.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 43 of its 69 references that are in the map. First posted on arXiv 23 Feb 2022.

Builds on

43 of 69 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2013Auto-Encoding Variational Bayescited by 4,053
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  5. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  6. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  7. 2018Disentangling by Factorisingcited by 616
  8. 2015Weight Uncertainty in Neural Networkscited by 554
  9. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  10. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  11. 2015Importance Weighted Autoencoderscited by 456
  12. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
31 more
  1. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  2. 2017VAE with a VampPriorcited by 248
  3. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  4. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  5. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  6. 2019A Primer on PAC-Bayesian Learningcited by 132
  7. 2020Tighter risk certificates for neural networkscited by 112
  8. 2018Bias and Generalization in Deep Generative Models: An Empirical Studycited by 83
  9. 2020On the role of data in PAC-Bayes boundscited by 64
  10. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  11. 2018Structured Disentangled Representationscited by 62
  12. 2019Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysiscited by 59
  13. 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
  14. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 46
  15. 2020PAC-Bayes unleashed: generalisation bounds with unbounded lossescited by 43
  16. 2019Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networkscited by 38
  17. 2019PAC-Bayes Un-Expected Bernstein Inequalitycited by 36
  18. 2020On Implicit Regularization in β-VAEscited by 34
  19. 2018Amortized Inference Regularizationcited by 30
  20. 2020Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networkscited by 29
  21. 2020Some Theoretical Insights into Wasserstein GANscited by 28
  22. 2016Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayescited by 27
  23. 2020Statistical guarantees for generative models without dominationcited by 27
  24. 2021Learning PAC-Bayes Priors for Probabilistic Neural Networkscited by 21
  25. 2021On Margins and Derandomisation in PAC-Bayescited by 19
  26. 2019PAC-Bayes under potentially heavy tailscited by 16
  27. 2019Efron-Stein PAC-Bayesian Inequalitiescited by 16
  28. 2020Evaluating Lossy Compression Rates of Deep Generative Modelscited by 16
  29. 2019Generalization Bounds For Unsupervised and Semi-Supervised Learning With Autoencoderscited by 11
  30. 2019A Scale Invariant Flatness Measure for Deep Network Minimacited by 8
  31. 2019Dissecting Non-Vacuous Generalization Bounds based on the Mean-Field Approximationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 2; 2025: 4; 2026: 4.

20 papers.

Sort:

By area:

  1. 2022PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingalescited by 24
  2. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 21
  3. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  4. 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
  5. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
  6. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 12
  7. 2022Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distancescited by 7
  8. 2024Tight PAC-Bayesian Risk Certificates for Contrastive Learningcited by 6
  9. 2022Learning Latent Representations to Co-Adapt to Humanscited by 5
  10. 2025Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysiscited by 3
  11. 2025PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoderscited by 3
  12. 2023Federated Learning with Nonvacuous Generalisation Boundscited by 2
8 more
  1. 2026Bound to Disagree: Generalization Bounds via Certifiable Surrogatescited by 2
  2. 2022On the Equality of the ELBO to a Sum of Entropies at Stationary Points of Learningcited by 1
  3. 2023A note on regularised NTK dynamics with an application to PAC-Bayesian trainingcited by 1
  4. 2025Multiview Representation Learning via Distributed Joint Latent Space Structuringcited by 1
  5. 2026PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoderscited by 1
  6. 2025Information-theoretic Generalization Analysis for VQ-VAEs: A Role of Latent Variablescited by 0
  7. 2026Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Trainingcited by 0
  8. 2026Latent Inference-Time Guidance of Time Series Foundation Modelscited by 0