# Batch Active Learning at Scale

arXiv: https://arxiv.org/abs/2107.14263 · HTML: https://mlsanity.com/paper/2107.14263/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [On Exact Computation with an Infinitely Wide Neural Net](https://mlsanity.com/paper/1904.11955/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning](https://mlsanity.com/paper/2012.10630/) (2020)
- [Lazier Than Lazy Greedy](https://mlsanity.com/paper/1409.7938/) (2014)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Task-Aware Variational Adversarial Active Learning](https://mlsanity.com/paper/2002.04709/) (2020)
- [Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details](https://mlsanity.com/paper/2102.01066/) (2021)
- [Data Shapley Valuation for Efficient Batch Active Learning](https://mlsanity.com/paper/2104.08312/) (2021)
- [Improved Algorithms for Efficient Active Learning Halfspaces with Massart and Tsybakov noise](https://mlsanity.com/paper/2102.05312/) (2021)
- [Scaling Hierarchical Agglomerative Clustering to Billion-sized Datasets](https://mlsanity.com/paper/2105.11653/) (2021)
- [Ranking CGANs: Subjective Control over Semantic Image Attributes](https://mlsanity.com/paper/1804.04082/) (2018)

## Built on by (most cited first, 30 of 127)

- [ShieldGemma: Generative AI Content Moderation Based on Gemma](https://mlsanity.com/paper/2407.21772/) (2024)
- [Coverage-centric Coreset Selection for High Pruning Rates](https://mlsanity.com/paper/2210.15809/) (2022)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning](https://mlsanity.com/paper/2311.08182/) (2023)
- [Reinforcement Learning from Human Feedback with Active Queries](https://mlsanity.com/paper/2402.09401/) (2024)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [GALAXY: Graph-based Active Learning at the Extreme](https://mlsanity.com/paper/2202.01402/) (2022)
- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Active Learning at the ImageNet Scale](https://mlsanity.com/paper/2111.12880/) (2021)
- [Plug and Play Active Learning for Object Detection](https://mlsanity.com/paper/2211.11612/) (2022)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [Exploring Active 3D Object Detection from a Generalization Perspective](https://mlsanity.com/paper/2301.09249/) (2023)
- [Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment](https://mlsanity.com/paper/2301.10625/) (2023)
- [Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning](https://mlsanity.com/paper/2210.07805/) (2022)
- [A Lagrangian Duality Approach to Active Learning](https://mlsanity.com/paper/2202.04108/) (2022)
- [An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models](https://mlsanity.com/paper/2401.06692/) (2024)
- [Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy](https://mlsanity.com/paper/2203.07845/) (2022)
- [ScatterShot: Interactive In-context Example Curation for Text Transformation](https://mlsanity.com/paper/2302.07346/) (2023)
- [LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning](https://mlsanity.com/paper/2306.09910/) (2023)
- [Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracy](https://mlsanity.com/paper/2311.01002/) (2023)
- [From Selection to Generation: A Survey of LLM-based Active Learning](https://mlsanity.com/paper/2502.11767/) (2025)
- [Algorithm Selection for Deep Active Learning with Imbalanced Datasets](https://mlsanity.com/paper/2302.07317/) (2023)
- [Fast Rates in Pool-Based Batch Active Learning](https://mlsanity.com/paper/2202.05448/) (2022)
- [Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyond](https://mlsanity.com/paper/2402.17327/) (2024)
- [Deep Active Learning with Noise Stability](https://mlsanity.com/paper/2205.13340/) (2022)
- [Active Learning with Neural Networks: Insights from Nonparametric Statistics](https://mlsanity.com/paper/2210.08367/) (2022)
- [KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection](https://mlsanity.com/paper/2307.07942/) (2023)

## Abstract (arXiv)

The ability to train complex and highly effective models often requires an abundance of training data, which can easily become a bottleneck in cost, time, and computational resources. Batch active learning, which adaptively issues batched queries to a labeling oracle, is a common approach for addressing this problem. The practical benefits of batch sampling come with the downside of less adaptivity and the risk of sampling redundant examples within a batch -- a risk that grows with the batch size. In this work, we analyze an efficient active learning algorithm, which focuses on the large batch setting. In particular, we show that our sampling method, which combines notions of uncertainty and diversity, easily scales to batch sizes (100K-1M) several orders of magnitude larger than used in previous studies and provides significant improvements in model training efficiency compared to recent baselines. Finally, we provide an initial theoretical analysis, proving label complexity guarantees for a related sampling method, which we show is approximately equivalent to our sampling method in specific settings.