# DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks

arXiv: https://arxiv.org/abs/1906.04569 · HTML: https://mlsanity.com/paper/1906.04569/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 30 of 92)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?](https://mlsanity.com/paper/2209.03302/) (2022)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglement](https://mlsanity.com/paper/2204.09308/) (2022)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [Second-Order Uncertainty Quantification: A Distance-Based Approach](https://mlsanity.com/paper/2312.00995/) (2023)
- [Learn to Accumulate Evidence from All Training Samples: Theory and Practice](https://mlsanity.com/paper/2306.11113/) (2023)
- [Quantification of Uncertainty with Adversarial Models](https://mlsanity.com/paper/2307.03217/) (2023)
- [GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection](https://mlsanity.com/paper/2104.14528/) (2021)
- [ValUES: A Framework for Systematic Validation of Uncertainty Estimation in Semantic Segmentation](https://mlsanity.com/paper/2401.08501/) (2024)
- [Label-wise Aleatoric and Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2406.02354/) (2024)
- [Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules](https://mlsanity.com/paper/2404.12215/) (2024)
- [Second-Order Uncertainty Quantification: Variance-Based Measures](https://mlsanity.com/paper/2401.00276/) (2023)
- [Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference](https://mlsanity.com/paper/2107.10384/) (2021)
- [Uncertainty-aware Language Modeling for Selective Question Answering](https://mlsanity.com/paper/2311.15451/) (2023)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning](https://mlsanity.com/paper/2406.00332/) (2024)
- [A Quantitative Comparison of Epistemic Uncertainty Maps Applied to Multi-Class Segmentation](https://mlsanity.com/paper/2109.10702/) (2021)
- [Benchmark for Out-of-Distribution Detection in Deep Reinforcement Learning](https://mlsanity.com/paper/2112.02694/) (2021)
- [GA-SmaAt-GNet: Generative Adversarial Small Attention GNet for Extreme Precipitation Nowcasting](https://mlsanity.com/paper/2401.09881/) (2024)
- [Harnessing Uncertainty-aware Bounding Boxes for Unsupervised 3D Object Detection](https://mlsanity.com/paper/2408.00619/) (2024)
- [Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation](https://mlsanity.com/paper/2412.07169/) (2024)
- [Learning Credal Ensembles via Distributionally Robust Optimization](https://mlsanity.com/paper/2602.08470/) (2026)
- [Improving the Reliability of Semantic Segmentation of Medical Images by Uncertainty Modeling with Bayesian Deep Networks and Curriculum Learning](https://mlsanity.com/paper/2108.11693/) (2021)
- [Scale-Dropout: Estimating Uncertainty in Deep Neural Networks Using Stochastic Scale](https://mlsanity.com/paper/2311.15816/) (2023)
- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Spatially-Aware Evaluation of Segmentation Uncertainty](https://mlsanity.com/paper/2506.16589/) (2025)
- [Bayesian posterior approximation with stochastic ensembles](https://mlsanity.com/paper/2212.08123/) (2022)
- [Uncertainty Quantification for Gradient-based Explanations in Neural Networks](https://mlsanity.com/paper/2403.17224/) (2024)
- [Stochasticity in Motion: An Information-Theoretic Approach to Trajectory Prediction](https://mlsanity.com/paper/2410.01628/) (2024)

## Abstract (arXiv)

Deep neural networks (DNNs) have achieved state-of-the-art performances in many important domains, including medical diagnosis, security, and autonomous driving. In these domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications. Bayesian neural networks attempt to address this challenge. However, traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method, called MC-DropConnect, gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify the uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art.