# Uncertainty Herding: One Active Learning Method for All Label Budgets

arXiv: https://arxiv.org/abs/2412.20644 · HTML: https://mlsanity.com/paper/2412.20644/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 30 Dec 2024.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [Active Learning Through a Covering Lens](https://mlsanity.com/paper/2205.11320/) (2022)
- [Active Learning by Feature Mixing](https://mlsanity.com/paper/2203.07034/) (2022)
- [Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks](https://mlsanity.com/paper/2002.06753/) (2020)
- [Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithms](https://mlsanity.com/paper/1810.05691/) (2018)
- [Batch Active Learning Using Determinantal Point Processes](https://mlsanity.com/paper/1906.07975/) (2019)
- [Fast and Eager k-Medoids Clustering: O(k) Runtime Improvement of the PAM, CLARA, and CLARANS Algorithms](https://mlsanity.com/paper/2008.05171/) (2020)
- [Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm](https://mlsanity.com/paper/2303.14382/) (2023)
- [Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach](https://mlsanity.com/paper/2106.02968/) (2021)
- [Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels](https://mlsanity.com/paper/2206.12569/) (2022)
- [Active and Continuous Exploration with Deep Neural Networks and Expected Model Output Changes](https://mlsanity.com/paper/1612.06129/) (2016)
- [How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget](https://mlsanity.com/paper/2306.03543/) (2023)
- [Generalized Coverage for More Robust Low-Budget Active Learning](https://mlsanity.com/paper/2407.12212/) (2024)
- [TiDAL: Learning Training Dynamics for Active Learning](https://mlsanity.com/paper/2210.06788/) (2022)
- [Active Distance-Based Clustering using K-medoids](https://mlsanity.com/paper/1512.03953/) (2015)
- [Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training](https://mlsanity.com/paper/2403.03728/) (2024)

## Built on by (most cited first, 15 of 15)

- [Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning](https://mlsanity.com/paper/2511.22344/) (2025)
- [A Mechanism-Driven Theory of Phase Transitions in Active Learning](https://mlsanity.com/paper/2607.00144/) (2026)
- [Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification](https://mlsanity.com/paper/2509.24181/) (2025)
- [Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation](https://mlsanity.com/paper/2510.22229/) (2025)
- [Conformal Cross-Modal Active Learning](https://mlsanity.com/paper/2603.23159/) (2026)
- [Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs](https://mlsanity.com/paper/2604.02766/) (2026)
- [Active Learning for Cascaded Object Detection: Balancing Coverage and Uncertainty in Table Extraction Pipelines](https://mlsanity.com/paper/2607.00747/) (2026)
- [One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning](https://mlsanity.com/paper/2608.03249/) (2026)
- [How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification](https://mlsanity.com/paper/2608.03511/) (2026)
- [Hard Cases, Bad Labels: Testing Error Exposure and Error Location in Uncertainty Sampling Under Bounded Label Noise](https://mlsanity.com/paper/2608.13601/) (2026)
- [SUGFW+: An Uncertainty-guided Feature Weighting Framework for Cold Start Active Adaptation of SAM in Medical Image Segmentation](https://mlsanity.com/paper/2608.16110/) (2026)
- [Probing Association Instability with Track-State Perturbations for Clip-Level Active Learning in Query-Propagation Multi-Object Tracking](https://mlsanity.com/paper/2608.17224/) (2026)
- [FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shifts](https://mlsanity.com/paper/2609.31637/) (2026)
- [CAENL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control](https://mlsanity.com/paper/2610.17509/) (2026)
- [Improving Category-Aware Active Domain Adaptation via Uncertainty-Guided Region Coverage](https://mlsanity.com/paper/2610.19803/) (2026)

## Abstract (arXiv)

Most active learning research has focused on methods which perform well when many labels are available, but can be dramatically worse than random selection when label budgets are small. Other methods have focused on the low-budget regime, but do poorly as label budgets increase. As the line between "low" and "high" budgets varies by problem, this is a serious issue in practice. We propose uncertainty coverage, an objective which generalizes a variety of low- and high-budget objectives, as well as natural, hyperparameter-light methods to smoothly interpolate between low- and high-budget regimes. We call greedy optimization of the estimate Uncertainty Herding; this simple method is computationally fast, and we prove that it nearly optimizes the distribution-level coverage. In experimental validation across a variety of active learning tasks, our proposal matches or beats state-of-the-art performance in essentially all cases; it is the only method of which we are aware that reliably works well in both low- and high-budget settings.