# Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks

arXiv: https://arxiv.org/abs/2211.12717 · HTML: https://mlsanity.com/paper/2211.12717/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 72 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding](https://mlsanity.com/paper/1511.02680/) (2015)
- [Understanding Measures of Uncertainty for Adversarial Example Detection](https://mlsanity.com/paper/1803.08533/) (2018)
- [Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors](https://mlsanity.com/paper/2005.07186/) (2020)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation](https://mlsanity.com/paper/1808.01200/) (2018)
- [Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?](https://mlsanity.com/paper/2006.14911/) (2020)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.10671/) (2020)
- [Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness](https://mlsanity.com/paper/1905.13472/) (2019)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Uncertainty-Aware Reinforcement Learning for Collision Avoidance](https://mlsanity.com/paper/1702.01182/) (2017)
- [Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning](https://mlsanity.com/paper/2106.04015/) (2021)

## Built on by (most cited first, 30 of 49)

- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations](https://mlsanity.com/paper/2404.10960/) (2024)
- [MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs](https://mlsanity.com/paper/2402.11756/) (2024)
- [Tractable Function-Space Variational Inference in Bayesian Neural Networks](https://mlsanity.com/paper/2312.17199/) (2023)
- [Quantification of Uncertainty with Adversarial Models](https://mlsanity.com/paper/2307.03217/) (2023)
- [Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty](https://mlsanity.com/paper/2311.08309/) (2023)
- [A Primer on Bayesian Neural Networks: Review and Debates](https://mlsanity.com/paper/2309.16314/) (2023)
- [CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks](https://mlsanity.com/paper/2401.05043/) (2024)
- [Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift](https://mlsanity.com/paper/2306.12306/) (2023)
- [Function-Space Regularization in Neural Networks: A Probabilistic Perspective](https://mlsanity.com/paper/2312.17162/) (2023)
- [Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generation](https://mlsanity.com/paper/2406.01806/) (2024)
- [Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise](https://mlsanity.com/paper/2301.01054/) (2023)
- [(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Models](https://mlsanity.com/paper/2409.02628/) (2024)
- [Two Sides of Miscalibration: Identifying Over and Under-Confidence Prediction for Network Calibration](https://mlsanity.com/paper/2308.03172/) (2023)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Out-of-Distribution Detection Methods Answer the Wrong Questions](https://mlsanity.com/paper/2507.01831/) (2025)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics](https://mlsanity.com/paper/2410.19780/) (2024)
- [Context-Guided Diffusion for Out-of-Distribution Molecular and Protein Design](https://mlsanity.com/paper/2407.11942/) (2024)
- [Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification](https://mlsanity.com/paper/2604.17112/) (2026)
- [Tools in the Loop: Quantifying Uncertainty of LLM Question Answering Systems That Use Tools](https://mlsanity.com/paper/2505.16113/) (2025)
- [BeliefPPG: Uncertainty-aware Heart Rate Estimation from PPG signals via Belief Propagation](https://mlsanity.com/paper/2306.07730/) (2023)
- [Informative Priors Improve the Reliability of Multimodal Clinical Data Classification](https://mlsanity.com/paper/2312.00794/) (2023)
- [SafeGround: Know When to Trust GUI Grounding Models via Uncertainty Calibration](https://mlsanity.com/paper/2602.02419/) (2026)
- [Understanding Silent Failures in Medical Image Classification](https://mlsanity.com/paper/2307.14729/) (2023)
- [Empirical Optimal Risk to Quantify Model Trustworthiness for Failure Detection](https://mlsanity.com/paper/2308.03179/) (2023)
- [CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMs](https://mlsanity.com/paper/2406.03158/) (2024)
- [TapType: Ten-finger text entry on everyday surfaces via Bayesian inference](https://mlsanity.com/paper/2410.06001/) (2024)
- [Arctique: An artificial histopathological dataset unifying realism and controllability for uncertainty quantification](https://mlsanity.com/paper/2411.07097/) (2024)

## Abstract (arXiv)

Bayesian deep learning seeks to equip deep neural networks with the ability to precisely quantify their predictive uncertainty, and has promised to make deep learning more reliable for safety-critical real-world applications. Yet, existing Bayesian deep learning methods fall short of this promise; new methods continue to be evaluated on unrealistic test beds that do not reflect the complexities of downstream real-world tasks that would benefit most from reliable uncertainty quantification. We propose the RETINA Benchmark, a set of real-world tasks that accurately reflect such complexities and are designed to assess the reliability of predictive models in safety-critical scenarios. Specifically, we curate two publicly available datasets of high-resolution human retina images exhibiting varying degrees of diabetic retinopathy, a medical condition that can lead to blindness, and use them to design a suite of automated diagnosis tasks that require reliable predictive uncertainty quantification. We use these tasks to benchmark well-established and state-of-the-art Bayesian deep learning methods on task-specific evaluation metrics. We provide an easy-to-use codebase for fast and easy benchmarking following reproducibility and software design principles. We provide implementations of all methods included in the benchmark as well as results computed over 100 TPU days, 20 GPU days, 400 hyperparameter configurations, and evaluation on at least 6 random seeds each.