mlsanityData to 7 Oct 2026

Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors

arXiv 1603.04733 ↗First posted 15 Mar 2016stat.ML cs.LGcited by 82

Not on the map. Why?

Abstract (arXiv)

We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices. Specifically, we employ a matrix variate Gaussian citegupta1999matrix parameter posterior distribution where we explicitly model the covariance among the input and output dimensions of each layer. Furthermore, with approximate covariance matrices we can achieve a more efficient way to represent those correlations that is also cheaper than fully factorized parameter posteriors. We further show that with the "local reprarametrization trick" citekingma2015variational on this posterior distribution we arrive at a Gaussian Process citerasmussen2006gaussian interpretation of the hidden units in each layer and we, similarly with citegal2015dropout, provide connections with deep Gaussian processes. We continue in taking advantage of this duality and incorporate "pseudo-data" citesnelson2005sparse in our model, which in turn allows for more efficient sampling while maintaining the properties of the original model. The validity of the proposed approach is verified through extensive experiments.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Mar 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 12; 2023: 22; 2024: 8; 2025: 8; 2026: 10.

82 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  3. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  4. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  5. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  6. 2021The Bayesian Learning Rulecited by 95
  7. 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
  8. 2021Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentationcited by 61
  9. 2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
  10. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  11. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  12. 2023BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inferencecited by 30
38 more
  1. 2022Accelerated Linearized Laplace Approximation for Bayesian Deep Learningcited by 27
  2. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  3. 2023A Primer on Bayesian Neural Networks: Review and Debatescited by 26
  4. 2021Robustness via Cross-Domain Ensemblescited by 23
  5. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  6. 2024Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Modelscited by 20
  7. 2022BayesFormer: Transformer with Uncertainty Estimationcited by 19
  8. 2023Uncertainty-Aware Distillation for Semi-Supervised Few-Shot Class-Incremental Learningcited by 16
  9. 2023The Geometry of Neural Nets' Parameter Spaces Under Reparametrizationcited by 16
  10. 2023Multiclass Confidence and Localization Calibration for Object Detectioncited by 16
  11. 2024A Review of Pseudo-Labeling for Computer Visioncited by 16
  12. 2023Improving Neural Additive Models with Bayesian Principlescited by 15
  13. 2022Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networkscited by 14
  14. 2021Pathologies in priors and inference for Bayesian transformerscited by 13
  15. 2021DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecastingcited by 11
  16. 2023Towards Accelerated Model Training via Bayesian Data Selectioncited by 11
  17. 2023Cal-DETR: Calibrated Detection Transformercited by 11
  18. 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
  19. 2022Task Agnostic and Post-hoc Unseen Distribution Detectioncited by 8
  20. 2022Are you using test log-likelihood correctly?cited by 8
  21. 2024Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolutioncited by 8
  22. 2021Structured Dropout Variational Inference for Bayesian Neural Networkscited by 7
  23. 2023Probabilistic Circuits That Know What They Don't Knowcited by 7
  24. 2021TyXe: Pyro-based Bayesian neural nets for Pytorchcited by 6
  25. 2022On the Importance of Gradient Norm in PAC-Bayesian Boundscited by 6
  26. 2024A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentationcited by 6
  27. 2021Introspective Robot Perception using Smoothed Predictions from Bayesian Neural Networkscited by 5
  28. 2021Uncertainty-Aware Semi-Supervised Few Shot Segmentationcited by 5
  29. 2022Controlled Dropout for Uncertainty Estimationcited by 4
  30. 2023Density Uncertainty Layers for Reliable Uncertainty Estimationcited by 4
  31. 2023Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networkscited by 4
  32. 2024Noether's razor: Learning Conserved Quantitiescited by 4
  33. 2025Tractable Representation Learning with Probabilistic Circuitscited by 4
  34. 2021W2WNet: a two-module probabilistic Convolutional Neural Network with embedded data cleansing functionalitycited by 3
  35. 2021Exploring Uncertainty in Deep Learning for Construction of Prediction Intervalscited by 3
  36. 2024Watch Your Head: Assembling Projection Heads to Save the Reliability of Federated Modelscited by 3
  37. 2021Analytically Tractable Bayesian Deep Q-Learningcited by 2
  38. 2023Uncertainty Propagation in Node Classificationcited by 2

50 most cited of 82 All 82 (JSON)