# Convergence of Uncertainty Sampling for Active Learning

arXiv: https://arxiv.org/abs/2110.15784 · HTML: https://mlsanity.com/paper/2110.15784/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 34 references that are in the map.
- First posted on arXiv 29 Oct 2021.

## Builds on (most cited first, 4 of 4)

- [Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron](https://mlsanity.com/paper/1810.07288/) (2018)
- [Active Learning Using Uncertainty Information](https://mlsanity.com/paper/1702.08540/) (2017)
- [Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss](https://mlsanity.com/paper/1812.01815/) (2018)
- [Interactive algorithms: from pool to stream](https://mlsanity.com/paper/1602.01132/) (2016)

## Built on by (most cited first, 19 of 19)

- [A Survey on Uncertainty Quantification Methods for Deep Learning](https://mlsanity.com/paper/2302.13425/) (2023)
- [Active Evaluation Acquisition for Efficient LLM Benchmarking](https://mlsanity.com/paper/2410.05952/) (2024)
- [Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation](https://mlsanity.com/paper/2202.06881/) (2022)
- [A Lagrangian Duality Approach to Active Learning](https://mlsanity.com/paper/2202.04108/) (2022)
- [Is margin all you need? An extensive empirical study of active learning on tabular data](https://mlsanity.com/paper/2210.03822/) (2022)
- [ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models](https://mlsanity.com/paper/2307.00398/) (2023)
- [Querying Easily Flip-flopped Samples for Deep Active Learning](https://mlsanity.com/paper/2401.09787/) (2024)
- [Understanding Uncertainty Sampling via Equivalent Loss](https://mlsanity.com/paper/2307.02719/) (2023)
- [User-Side Realization](https://mlsanity.com/paper/2403.15757/) (2024)
- [A Cost-Effective LLM-based Approach to Identify Wildlife Trafficking in Online Marketplaces](https://mlsanity.com/paper/2504.21211/) (2025)
- [MAViC: Multimodal Active Learning for Video Captioning](https://mlsanity.com/paper/2212.11109/) (2022)
- [On the Convergence of Loss and Uncertainty-based Active Learning Algorithms](https://mlsanity.com/paper/2312.13927/) (2023)
- [PH-Dropout: Practical Epistemic Uncertainty Quantification for View Synthesis](https://mlsanity.com/paper/2410.05468/) (2024)
- [Active Learning from the Web](https://mlsanity.com/paper/2210.08205/) (2022)
- [Improve Cost Efficiency of Active Learning over Noisy Dataset](https://mlsanity.com/paper/2403.01346/) (2024)
- [ADs: Active Data-sharing for Data Quality Assurance in Advanced Manufacturing Systems](https://mlsanity.com/paper/2404.00572/) (2024)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [Nearly Optimal Active Preference Learning and Its Application to LLM Alignment](https://mlsanity.com/paper/2602.01581/) (2026)
- [Balancing Uncertainty and Diversity of Samples: Leveraging Diversity of Least, High Confidence Samples for Effective Active Learning](https://mlsanity.com/paper/2605.22169/) (2026)

## Abstract (arXiv)

Uncertainty sampling in active learning is heavily used in practice to reduce the annotation cost. However, there has been no wide consensus on the function to be used for uncertainty estimation in binary classification tasks and convergence guarantees of the corresponding active learning algorithms are not well understood. The situation is even more challenging for multi-category classification. In this work, we propose an efficient uncertainty estimator for binary classification which we also extend to multiple classes, and provide a non-asymptotic rate of convergence for our uncertainty sampling-based active learning algorithm in both cases under no-noise conditions (i.e., linearly separable data). We also extend our analysis to the noisy case and provide theoretical guarantees for our algorithm under the influence of noise in the task of binary and multi-class classification.