mlsanityData to 7 Oct 2026

Subspace Inference for Bayesian Deep Learning

arXiv 1907.07504 ↗First posted 17 Jul 2019cs.LG stat.MLcited by 94

Not on the map. Why?

Abstract (arXiv)

Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty. However, scaling Bayesian inference techniques to deep neural networks is challenging due to the high dimensionality of the parameter space. In this paper, we construct low-dimensional subspaces of parameter space, such as the first principal components of the stochastic gradient descent (SGD) trajectory, which contain diverse sets of high performing models. In these subspaces, we are able to apply elliptical slice sampling and variational inference, which struggle in the full parameter space. We show that Bayesian model averaging over the induced posterior in these subspaces produces accurate predictions and well calibrated predictive uncertainty for both regression and image classification.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 17; 2023: 15; 2024: 18; 2025: 12; 2026: 9.

94 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  3. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  4. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  5. 2021Bayesian Neural Network Priors Revisitedcited by 121
  6. 2021Learning Neural Network Subspacescited by 82
  7. 2021Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensemblingcited by 75
  8. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  9. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  10. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  11. 2022The Unreasonable Effectiveness of Deep Evidential Regressioncited by 40
  12. 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
38 more
  1. 2023Merging by Matching Models in Task Parameter Subspacescited by 36
  2. 2021From partners to populations: A hierarchical Bayesian account of coordination and conventioncited by 27
  3. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  4. 2022On Calibrating Semantic Segmentation Models: Analyses and An Algorithmcited by 25
  5. 2021Multivariate Deep Evidential Regressioncited by 23
  6. 2023Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetrycited by 20
  7. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  8. 2023Variational Linearized Laplace Approximation for Bayesian Deep Learningcited by 17
  9. 2024Reparameterization invariance in approximate Bayesian inferencecited by 17
  10. 2021Sparse Uncertainty Representation in Deep Learning with Inducing Weightscited by 16
  11. 2023Uncertainty Quantification via Neural Posterior Principal Componentscited by 16
  12. 2021Efficient Online Bayesian Inference for Neural Banditscited by 15
  13. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  14. 2022Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertaintycited by 13
  15. 2021Connecting Low-Loss Subspace for Personalized Federated Learningcited by 12
  16. 2021Deep Bayesian inference for seismic imaging with taskscited by 12
  17. 2024Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Modelscited by 12
  18. 2025Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inferencecited by 11
  19. 2021Model Selection for Bayesian Autoencoderscited by 10
  20. 2023Langevin Monte Carlo for strongly log-concave distributions: Randomized midpoint revisitedcited by 10
  21. 2025Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networkscited by 10
  22. 2022Feature Space Particle Inference for Neural Network Ensemblescited by 9
  23. 2023Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networkscited by 9
  24. 2022Tackling covariate shift with node-based Bayesian neural networkscited by 8
  25. 2022Are you using test log-likelihood correctly?cited by 8
  26. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  27. 2023Input-gradient space particle inference for neural network ensemblescited by 7
  28. 2021Global optimization using random embeddingscited by 6
  29. 2023Collapsed Inference for Bayesian Deep Learningcited by 6
  30. 2025Paths and Ambient Spaces in Neural Loss Landscapescited by 6
  31. 2022What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problemscited by 5
  32. 2022Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generationcited by 5
  33. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  34. 2024Confidence-aware multi-modality learning for eye disease screeningcited by 5
  35. 2024Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learningcited by 5
  36. 2025CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Samplingcited by 5
  37. 2025Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRAcited by 5
  38. 2021Uncertainty-Aware Boosted Ensembling in Multi-Modal Settingscited by 4

50 most cited of 94 All 94 (JSON)