# Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning

arXiv: https://arxiv.org/abs/2106.12059 · HTML: https://mlsanity.com/paper/2106.12059/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 66 references that are in the map.
- First posted on arXiv 22 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [Deep Deterministic Uncertainty: A Simple Baseline](https://mlsanity.com/paper/2102.11582/) (2021)
- [A* Sampling](https://mlsanity.com/paper/1411.0030/) (2014)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [On Statistical Bias In Active Learning: How and When To Fix It](https://mlsanity.com/paper/2101.11665/) (2021)
- [Boltzmann Exploration Done Right](https://mlsanity.com/paper/1705.10257/) (2017)
- [Bayesian Batch Active Learning as Sparse Subset Approximation](https://mlsanity.com/paper/1908.02144/) (2019)
- [Batch Active Learning Using Determinantal Point Processes](https://mlsanity.com/paper/1906.07975/) (2019)
- [Bayesian active learning for production, a systematic study and a reusable library](https://mlsanity.com/paper/2006.09916/) (2020)
- [Can Active Learning Preemptively Mitigate Fairness Issues?](https://mlsanity.com/paper/2104.06879/) (2021)
- [Synbols: Probing Learning Algorithms with Synthetic Datasets](https://mlsanity.com/paper/2009.06415/) (2020)
- [Low-Regret Active learning](https://mlsanity.com/paper/2104.02822/) (2021)

## Built on by (most cited first, 30 of 36)

- [Rethinking Aleatoric and Epistemic Uncertainty](https://mlsanity.com/paper/2412.20892/) (2024)
- [Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data](https://mlsanity.com/paper/2111.02275/) (2021)
- [Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantities](https://mlsanity.com/paper/2208.00549/) (2022)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Deep Bayesian Active Learning for Preference Modeling in Large Language Models](https://mlsanity.com/paper/2406.10023/) (2024)
- [Multi-Fidelity Active Learning with GFlowNets](https://mlsanity.com/paper/2306.11715/) (2023)
- [Black-Box Batch Active Learning for Regression](https://mlsanity.com/paper/2302.08981/) (2023)
- [Making Better Use of Unlabelled Data in Bayesian Active Learning](https://mlsanity.com/paper/2404.17249/) (2024)
- [Querying Easily Flip-flopped Samples for Deep Active Learning](https://mlsanity.com/paper/2401.09787/) (2024)
- [Scaling Textual Gradients via Sampling-Based Momentum](https://mlsanity.com/paper/2506.00400/) (2025)
- [Fusion or Confusion? Multimodal Complexity Is Not All You Need](https://mlsanity.com/paper/2512.22991/) (2025)
- [Active Learning with Selective Time-Step Acquisition for PDEs](https://mlsanity.com/paper/2511.18107/) (2025)
- [Progressive Generalization Risk Reduction for Data-Efficient Causal Effect Estimation](https://mlsanity.com/paper/2411.11256/) (2024)
- [Enhancing Treatment Effect Estimation via Active Learning: A Counterfactual Covering Perspective](https://mlsanity.com/paper/2505.05242/) (2025)
- [Depth Uncertainty Networks for Active Learning](https://mlsanity.com/paper/2112.06796/) (2021)
- [PyRelationAL: a python library for active learning research and development](https://mlsanity.com/paper/2205.11117/) (2022)
- [Efficient Bayesian Updates for Deep Active Learning via Laplace Approximations](https://mlsanity.com/paper/2210.06112/) (2022)
- [Speeding Up BatchBALD: A k-BALD Family of Approximations for Active Learning](https://mlsanity.com/paper/2301.09490/) (2023)
- [Causal-EPIG: A Prediction-Oriented Active Learning Framework for CATE Estimation](https://mlsanity.com/paper/2509.21866/) (2025)
- [BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning](https://mlsanity.com/paper/2603.13109/) (2026)
- [Learning Where to Simulate: Generative Active Sampling for Online PDE Surrogate Training](https://mlsanity.com/paper/2606.09949/) (2026)
- [Information-Theoretic Active Correlation Clustering](https://mlsanity.com/paper/2402.03587/) (2024)
- [Active Learning to Guide Labeling Efforts for Question Difficulty Estimation](https://mlsanity.com/paper/2409.09258/) (2024)
- [MALADY: Multiclass Active Learning with Auction Dynamics on Graphs](https://mlsanity.com/paper/2409.09475/) (2024)
- [Learning Set Functions with Implicit Differentiation](https://mlsanity.com/paper/2412.11239/) (2024)
- [ActiveCQ: Active Estimation of Causal Quantities](https://mlsanity.com/paper/2509.24293/) (2025)
- [Towards Open-Ended Discovery for Low-Resource NLP](https://mlsanity.com/paper/2510.01220/) (2025)
- [Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging](https://mlsanity.com/paper/2601.13677/) (2026)
- [Active Learning Using Aggregated Acquisition Functions: Accuracy and Sustainability Analysis](https://mlsanity.com/paper/2602.07440/) (2026)
- [Observationally Informed Adaptive Causal Experimental Design](https://mlsanity.com/paper/2603.03785/) (2026)

## Abstract (arXiv)

We examine a simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning. Unlike acquiring the top-K points from the pool set, score- or rank-based sampling takes into account that acquisition scores change as new data are acquired. This simple strategy for adapting standard single-sample acquisition strategies can even perform just as well as compute-intensive state-of-the-art batch acquisition functions, like BatchBALD or BADGE, while using orders of magnitude less compute. In addition to providing a practical option for machine learning practitioners, the surprising success of the proposed method in a wide range of experimental settings raises a difficult question for the field: when are these expensive batch acquisition methods pulling their weight?