# Learnable Bernoulli Dropout for Bayesian Deep Learning

arXiv: https://arxiv.org/abs/2002.05155 · HTML: https://mlsanity.com/paper/2002.05155/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2020.

## Built on by (most cited first, 14 of 14)

- [Contextual Dropout: An Efficient Sample-Dependent Dropout Module](https://mlsanity.com/paper/2103.04181/) (2021)
- [Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers](https://mlsanity.com/paper/2205.14336/) (2022)
- [GFlowOut: Dropout with Generative Flow Networks](https://mlsanity.com/paper/2210.12928/) (2022)
- [Data-driven discovery of interpretable causal relations for deep learning material laws with uncertainty propagation](https://mlsanity.com/paper/2105.09980/) (2021)
- [Not All Attention Is All You Need](https://mlsanity.com/paper/2104.04692/) (2021)
- [On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction](https://mlsanity.com/paper/2310.11479/) (2023)
- [A Bayesian Neural Network based on Dropout Regulation](https://mlsanity.com/paper/2102.01968/) (2021)
- [Gated Compression Layers for Efficient Always-On Models](https://mlsanity.com/paper/2303.08970/) (2023)
- [Diffusion Boosted Trees](https://mlsanity.com/paper/2406.01813/) (2024)
- [Gated Linear Model induced U-net for surrogate modeling and uncertainty quantification](https://mlsanity.com/paper/2111.05123/) (2021)
- [VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognition](https://mlsanity.com/paper/2204.00130/) (2022)
- [Learnable Model Augmentation Self-Supervised Learning for Sequential Recommendation](https://mlsanity.com/paper/2204.10128/) (2022)
- [Motif Guided Graph Transformer with Combinatorial Skeleton Prototype Learning for Skeleton-Based Person Re-Identification](https://mlsanity.com/paper/2412.09044/) (2024)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)

## Abstract (arXiv)

In this work, we propose learnable Bernoulli dropout (LBD), a new model-agnostic dropout scheme that considers the dropout rates as parameters jointly optimized with other model parameters. By probabilistic modeling of Bernoulli dropout, our method enables more robust prediction and uncertainty quantification in deep models. Especially, when combined with variational auto-encoders (VAEs), LBD enables flexible semi-implicit posterior representations, leading to new semi-implicit VAE~(SIVAE) models. We solve the optimization for training with respect to the dropout parameters using Augment-REINFORCE-Merge (ARM), an unbiased and low-variance gradient estimator. Our experiments on a range of tasks show the superior performance of our approach compared with other commonly used dropout schemes. Overall, LBD leads to improved accuracy and uncertainty estimates in image classification and semantic segmentation. Moreover, using SIVAE, we can achieve state-of-the-art performance on collaborative filtering for implicit feedback on several public datasets.