mlsanityData to 14 Oct 2026

Convergence Rates of Variational Inference in Sparse Deep Learning

arXiv 1908.04847 ↗First posted 9 Aug 2019math.ST cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

Variational inference is becoming more and more popular for approximating intractable posterior distributions in Bayesian statistics and machine learning. Meanwhile, a few recent works have provided theoretical justification and new insights on deep neural networks for estimating smooth functions in usual settings such as nonparametric regression. In this paper, we show that variational inference for sparse deep learning retains the same generalization properties than exact Bayesian inference. In particular, we highlight the connection between estimation and approximation theories via the classical bias-variance trade-off and show that it leads to near-minimax rates of convergence for H"older smooth functions. Additionally, we show that the model selection framework over the neural network architecture via ELBO maximization does not overfit and adaptively achieves the optimal rate of convergence.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 8; 2024: 5; 2025: 3; 2026: 2.

27 papers.

Sort:

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  2. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  3. 2021Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Detailscited by 14
  4. 2024Posterior and variational inference for deep neural networks with heavy-tailed weightscited by 10
  5. 2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
  6. 2023FedHB: Hierarchical Bayesian Federated Learningcited by 8
  7. 2022On the inability of Gaussian process regression to optimally learn compositional functionscited by 7
  8. 2022Asymptotic Properties for Bayesian Neural Network in Besov Spacecited by 7
  9. 2022Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selectioncited by 7
  10. 2024Misclassification bounds for PAC-Bayesian sparse deep learningcited by 7
  11. 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
  12. 2023Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertaintycited by 5
15 more
  1. 2024Deep Horseshoe Gaussian Processescited by 5
  2. 2023FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropoutcited by 4
  3. 2023Spike-and-slab shrinkage priors for structurally sparse Bayesian neural networkscited by 4
  4. 2024Minimax optimality of deep neural networks on dependent data via PAC-Bayes boundscited by 4
  5. 2021A fast asynchronous MCMC sampler for sparse Bayesian inferencecited by 3
  6. 2025Optimization Guarantees for Square-Root Natural-Gradient Variational Inferencecited by 3
  7. 2022A PAC-Bayes oracle inequality for sparse neural networkscited by 2
  8. 2025Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionalitycited by 2
  9. 2026Bayesian Lottery Ticket Hypothesiscited by 2
  10. 2022Masked Bayesian Neural Networks : Computation and Optimalitycited by 1
  11. 2023Personalized Federated Learning via Amortized Bayesian Meta-Learningcited by 1
  12. 2023The surrogate Gibbs-posterior of a corrected stochastic MALA: Towards uncertainty quantification for neural networkscited by 0
  13. 2024Variational Bayesian Bow tie Neural Networks with Shrinkagecited by 0
  14. 2025SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributionscited by 0
  15. 2026Posterior Concentration of Bayesian Physics-Informed Neural Networks for Elliptic PDEscited by 0