# Deep Active Ensemble Sampling For Image Classification

arXiv: https://arxiv.org/abs/2210.05770 · HTML: https://mlsanity.com/paper/2210.05770/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Deep Exploration via Bootstrapped DQN](https://mlsanity.com/paper/1602.04621/) (2016)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Suggestive Annotation: A Deep Active Learning Framework for Biomedical Image Segmentation](https://mlsanity.com/paper/1706.04737/) (2017)
- [Knowledge Distillation Meets Self-Supervision](https://mlsanity.com/paper/2006.07114/) (2020)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost](https://mlsanity.com/paper/1910.07153/) (2019)
- [Ensemble Sampling](https://mlsanity.com/paper/1705.07347/) (2017)
- [State-Relabeling Adversarial Active Learning](https://mlsanity.com/paper/2004.04943/) (2020)
- [Reducing Label Effort: Self-Supervised meets Active Learning](https://mlsanity.com/paper/2108.11458/) (2021)
- [Active Learning for Convolutional Neural Networks: A Core-Set Approach](https://mlsanity.com/paper/1708.00489/) (2017)
- [Distill on the Go: Online knowledge distillation in self-supervised learning](https://mlsanity.com/paper/2104.09866/) (2021)
- [Minimax Active Learning](https://mlsanity.com/paper/2012.10467/) (2020)
- [Deep Bayesian Active Learning, A Brief Survey on Recent Advances](https://mlsanity.com/paper/2012.08044/) (2020)
- [Adaptive Region-Based Active Learning](https://mlsanity.com/paper/2002.07348/) (2020)
- [Deep GAN-Based Cross-Spectral Cross-Resolution Iris Recognition](https://mlsanity.com/paper/2108.01569/) (2021)
- [Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lens](https://mlsanity.com/paper/1309.6830/) (2013)
- [Visual Object Tracking based on Adaptive Siamese and Motion Estimation Network](https://mlsanity.com/paper/1810.00119/) (2018)

## Built on by (most cited first, 7 of 7)

- [More Synergy, Less Redundancy: Exploiting Joint Mutual Information for Self-Supervised Learning](https://mlsanity.com/paper/2307.00651/) (2023)
- [Making Better Use of Unlabelled Data in Bayesian Active Learning](https://mlsanity.com/paper/2404.17249/) (2024)
- [FUSSL: Fuzzy Uncertain Self Supervised Learning](https://mlsanity.com/paper/2210.15818/) (2022)
- [Multi-spectral Entropy Constrained Neural Compression of Solar Imagery](https://mlsanity.com/paper/2309.10791/) (2023)
- [Direct Coloring for Self-Supervised Enhanced Feature Decoupling](https://mlsanity.com/paper/2412.02109/) (2024)
- [Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition](https://mlsanity.com/paper/2412.02121/) (2024)
- [GUESS: Generative Uncertainty Ensemble for Self Supervision](https://mlsanity.com/paper/2412.02896/) (2024)

## Abstract (arXiv)

Conventional active learning (AL) frameworks aim to reduce the cost of data annotation by actively requesting the labeling for the most informative data points. However, introducing AL to data hungry deep learning algorithms has been a challenge. Some proposed approaches include uncertainty-based techniques, geometric methods, implicit combination of uncertainty-based and geometric approaches, and more recently, frameworks based on semi/self supervised techniques. In this paper, we address two specific problems in this area. The first is the need for efficient exploitation/exploration trade-off in sample selection in AL. For this, we present an innovative integration of recent progress in both uncertainty-based and geometric frameworks to enable an efficient exploration/exploitation trade-off in sample selection strategy. To this end, we build on a computationally efficient approximate of Thompson sampling with key changes as a posterior estimator for uncertainty representation. Our framework provides two advantages: (1) accurate posterior estimation, and (2) tune-able trade-off between computational overhead and higher accuracy. The second problem is the need for improved training protocols in deep AL. For this, we use ideas from semi/self supervised learning to propose a general approach that is independent of the specific AL technique being used. Taken these together, our framework shows a significant improvement over the state-of-the-art, with results that are comparable to the performance of supervised-learning under the same setting. We show empirical results of our framework, and comparative performance with the state-of-the-art on four datasets, namely, MNIST, CIFAR10, CIFAR100 and ImageNet to establish a new baseline in two different settings.