mlsanityData to 7 Oct 2026

Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

arXiv 1506.02142 ↗First posted 6 Jun 2015stat.ML cs.LGcited by 4,416

Not on the map. Why?

Cited by 79 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning tools have gained tremendous attention in applied machine learning. However such tools for regression and classification do not capture model uncertainty. In comparison, Bayesian models offer a mathematically grounded framework to reason about model uncertainty, but usually come with a prohibitive computational cost. In this paper we develop a new theoretical framework casting dropout training in deep neural networks (NNs) as approximate Bayesian inference in deep Gaussian processes. A direct result of this theory gives us tools to model uncertainty with dropout NNs -- extracting information from existing models that has been thrown away so far. This mitigates the problem of representing uncertainty in deep learning without sacrificing either computational complexity or test accuracy. We perform an extensive study of the properties of dropout's uncertainty. Various network architectures and non-linearities are assessed on tasks of regression and classification, using MNIST as an example. We show a considerable improvement in predictive log-likelihood and RMSE compared to existing state-of-the-art methods, and finish by using dropout's uncertainty in deep reinforcement learning.

Key facts. Cited by 4,416 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 4; 2019: 6; 2020: 8; 2021: 702; 2022: 693; 2023: 733; 2024: 740; 2025: 745; 2026: 785.

4,416 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  4. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  5. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  6. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  7. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  8. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  9. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  10. 2021Score-based diffusion models for accelerated MRIcited by 308
  11. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  12. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
38 more
  1. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  2. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  3. 2021Towards Open World Object Detectioncited by 297
  4. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  5. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  6. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  7. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  8. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  9. 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 237
  10. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  11. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  12. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  13. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  14. 2021Diffusion Models for Implicit Image Segmentation Ensemblescited by 198
  15. 2023Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hackingcited by 195
  16. 2021ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentationcited by 184
  17. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  18. 2022Biological Sequence Design with GFlowNetscited by 172
  19. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
  20. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  21. 2023Speculative Decoding with Big Little Decodercited by 163
  22. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  23. 2021Active Learning by Acquiring Contrastive Examplescited by 154
  24. 2021A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challengescited by 154
  25. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  26. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  27. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  28. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  29. 2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
  30. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  31. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  32. 2021OpenGAN: Open-Set Recognition via Open Data Generationcited by 135
  33. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  34. 2023Non-Parametric Outlier Synthesiscited by 129
  35. 2023End-to-End Speech Recognition: A Surveycited by 127
  36. 2021Uncertainty Weighted Actor-Critic for Offline Reinforcement Learningcited by 125
  37. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  38. 2022ActiveNeRF: Learning where to See with Uncertainty Estimationcited by 124

50 most cited of 4,416 All 4,416 (JSON)