# Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles

arXiv: https://arxiv.org/abs/1811.03897 · HTML: https://mlsanity.com/paper/1811.03897/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2018.

## Built on by (most cited first, 20 of 20)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [A Framework and Benchmark for Deep Batch Active Learning for Regression](https://mlsanity.com/paper/2203.09410/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Neural Active Learning with Performance Guarantees](https://mlsanity.com/paper/2106.03243/) (2021)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling](https://mlsanity.com/paper/2210.05043/) (2022)
- [Neural Active Learning Beyond Bandits](https://mlsanity.com/paper/2404.12522/) (2024)
- [Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentation](https://mlsanity.com/paper/2411.15763/) (2024)
- [Bayesian Estimate of Mean Proper Scores for Diversity-Enhanced Active Learning](https://mlsanity.com/paper/2312.10116/) (2023)
- [Image Quality Assessment: Integrating Model-Centric and Data-Centric Approaches](https://mlsanity.com/paper/2207.14769/) (2022)
- [Active Learning with Fully Bayesian Neural Networks for Discontinuous and Nonstationary Data](https://mlsanity.com/paper/2405.09817/) (2024)
- [Streamlining the Development of Active Learning Methods in Real-World Object Detection](https://mlsanity.com/paper/2508.19906/) (2025)
- [MDS-VQA: Model-Informed Data Selection for Video Quality Assessment](https://mlsanity.com/paper/2603.11525/) (2026)
- [Knowledge-driven Active Learning](https://mlsanity.com/paper/2110.08265/) (2021)
- [Through-life Monitoring of Resource-constrained Systems and Fleets](https://mlsanity.com/paper/2301.01017/) (2023)
- [Evaluating the effect of data augmentation and BALD heuristics on distillation of Semantic-KITTI dataset](https://mlsanity.com/paper/2302.10679/) (2023)
- [Stein Variational Newton Neural Network Ensembles](https://mlsanity.com/paper/2411.01887/) (2024)
- [When Imbalance Comes Twice: Active Learning under Simulated Class Imbalance and Label Shift in Binary Semantic Segmentation](https://mlsanity.com/paper/2601.06209/) (2026)
- [Beyond a Single Signal: SPECTREG2, A Unified MultiExpert Anomaly Detector for Unknown Unknowns](https://mlsanity.com/paper/2603.21160/) (2026)
- [VOLTA: The Surprising Ineffectiveness of Auxiliary Losses for Calibrated Deep Learning](https://mlsanity.com/paper/2604.08639/) (2026)

## Abstract (arXiv)

In image classification tasks, the ability of deep CNNs to deal with complex image data has proven to be unrivalled. However, they require large amounts of labeled training data to reach their full potential. In specialised domains such as healthcare, labeled data can be difficult and expensive to obtain. Active Learning aims to alleviate this problem, by reducing the amount of labelled data needed for a specific task while delivering satisfactory performance. We propose DEBAL, a new active learning strategy designed for deep neural networks. This method improves upon the current state-of-the-art deep Bayesian active learning method, which suffers from the mode collapse problem. We correct for this deficiency by making use of the expressive power and statistical properties of model ensembles. Our proposed method manages to capture superior data uncertainty, which translates into improved classification performance. We demonstrate empirically that our ensemble method yields faster convergence of CNNs trained on the MNIST and CIFAR-10 datasets.