Data to 7 Oct 2026

Concrete Dropout

arXiv 1705.07832 ↗First posted 22 May 2017stat.MLcited by 227

Not on the map. Why?

Abstract (arXiv)

Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessary - a prohibitive operation with large models, and an impossible one with RL. We propose a new dropout variant which gives improved performance and better calibrated uncertainties. Relying on recent developments in Bayesian deep learning, we use a continuous relaxation of dropout's discrete masks. Together with a principled optimisation objective, this allows for automatic tuning of the dropout probability in large models, and as a result faster experimentation cycles. In RL this allows the agent to adapt its uncertainty dynamically as more data is observed. We analyse the proposed variant extensively on a range of tasks, and give insights into common practice in the field where larger dropout probabilities are often used in deeper model layers.

Key facts. Cited by 227 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 0; 2020: 0; 2021: 60; 2022: 44; 2023: 49; 2024: 26; 2025: 32; 2026: 14.

227 papers.

Sort:

By area:

2021A Survey of Uncertainty in Deep Neural Networkscited by 675
2018Scalable agent alignment via reward modeling: a research directioncited by 438
2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
2021Machine Learning with a Reject Option: A surveycited by 107
2022CARD: Classification and Regression Diffusion Modelscited by 102
2021The Bayesian Learning Rulecited by 95
2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
2023DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Modelscited by 81
2022Source-Free Unsupervised Domain Adaptation: A Surveycited by 79
2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 72

38 more

2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
2021Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanationscited by 56
2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
2022Uncertainty-guided Source-free Domain Adaptationcited by 46
2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
2021Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classificationcited by 39
2022Robust Calibration with Multi-domain Temperature Scalingcited by 39
2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
2023Mind the Label Shift of Augmentation-based Graph OOD Generalizationcited by 34
2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
2022Copula Conformal Prediction for Multi-step Time Series Forecastingcited by 28
2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
2021Contextual Dropout: An Efficient Sample-Dependent Dropout Modulecited by 26
2021A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applicationscited by 24
2021Taxonomy of Machine Learning Safety: A Survey and Primercited by 22
2022GFlowOut: Dropout with Generative Flow Networkscited by 22
2023Dropout Reduces Underfittingcited by 21
2023Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shiftcited by 20
2021Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertaintycited by 19
2021Towards Lower-Dose PET using Physics-Based Uncertainty-Aware Multimodal Learning with Robustness to Out-of-Distribution Datacited by 18
2021Propagating State Uncertainty Through Trajectory Forecastingcited by 17
2023Optimal Training of Mean Variance Estimation Neural Networkscited by 17
2021How to Evaluate Uncertainty Estimates in Machine Learning for Regression?cited by 15
2023Dropout Injection at Test Time for Post Hoc Uncertainty Quantification in Neural Networkscited by 14
2021Pathologies in priors and inference for Bayesian transformerscited by 13
2024Scalable Bayesian Learning with posteriorscited by 13
2018Safe end-to-end imitation learning for model predictive controlcited by 12
2023Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shiftscited by 12
2024Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Modelscited by 12
2025Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splattingcited by 12
2021STUaNet: Understanding uncertainty in spatiotemporal collective human mobilitycited by 11

50 most cited of 227 All 227 (JSON)