mlsanityData to 7 Oct 2026

Weight Uncertainty in Neural Networks

arXiv 1505.05424 ↗First posted 20 May 2015stat.ML cs.LGcited by 554

Not on the map. Why?

Abstract (arXiv)

We introduce a new, efficient, principled and backpropagation-compatible algorithm for learning a probability distribution on the weights of a neural network, called Bayes by Backprop. It regularises the weights by minimising a compression cost, known as the variational free energy or the expected lower bound on the marginal likelihood. We show that this principled kind of regularisation yields comparable performance to dropout on MNIST classification. We then demonstrate how the learnt uncertainty in the weights can be used to improve generalisation in non-linear regression problems, and how this weight uncertainty can be used to drive the exploration-exploitation trade-off in reinforcement learning.

Key facts. Cited by 554 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 May 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 3; 2021: 149; 2022: 94; 2023: 86; 2024: 54; 2025: 87; 2026: 78.

554 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  5. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  6. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  7. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  8. 2021Epistemic Neural Networkscited by 120
  9. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  10. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  11. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  12. 2021The Bayesian Learning Rulecited by 95
38 more
  1. 2023Bayes' Rays: Uncertainty Quantification for Neural Radiance Fieldscited by 90
  2. 2021Learning Neural Network Subspacescited by 82
  3. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  4. 2022Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantificationcited by 71
  5. 2022Personalized Federated Learning via Variational Bayesian Inferencecited by 68
  6. 2024Variational Learning is Effective for Large Deep Networkscited by 62
  7. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  8. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55
  9. 2022Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Taskscited by 49
  10. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  11. 2022Sample-dependent Adaptive Temperature Scaling for Improved Calibrationcited by 48
  12. 2022ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpointscited by 48
  13. 2022Criteria for Classifying Forecasting Methodscited by 48
  14. 2021Posterior Meta-Replay for Continual Learningcited by 45
  15. 2022Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximationscited by 42
  16. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  17. 2022Adversarial Robustness of Deep Neural Networks: A Survey from a Formal Verification Perspectivecited by 41
  18. 2023Tractable Function-Space Variational Inference in Bayesian Neural Networkscited by 41
  19. 2021Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentationcited by 40
  20. 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
  21. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  22. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  23. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  24. 2021Is MC Dropout Bayesian?cited by 35
  25. 2022Uncertainty Estimation for Language Reward Modelscited by 35
  26. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  27. 2022Probabilistic Representations for Video Contrastive Learningcited by 33
  28. 2021Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networkscited by 32
  29. 2021Continual Learning via Bit-Level Information Preservingcited by 32
  30. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  31. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  32. 2021Bayesian Uncertainty Estimation of Learned Variational MRI Reconstructioncited by 30
  33. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  34. 2022Uncertainty-Aware Multi-View Representation Learningcited by 29
  35. 2022Adapting the Linearised Laplace Model Evidence for Modern Deep Learningcited by 29
  36. 2023Quantification of Uncertainty with Adversarial Modelscited by 28
  37. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  38. 2021A Bit More Bayesian: Domain-Invariant Learning with Uncertaintycited by 27

50 most cited of 554 All 554 (JSON)