# Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss

arXiv: https://arxiv.org/abs/1812.01815 · HTML: https://mlsanity.com/paper/1812.01815/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2018.

## Built on by (most cited first, 10 of 10)

- [A Lagrangian Duality Approach to Active Learning](https://mlsanity.com/paper/2202.04108/) (2022)
- [Convergence of Uncertainty Sampling for Active Learning](https://mlsanity.com/paper/2110.15784/) (2021)
- [Improved Algorithms for Agnostic Pool-based Active Classification](https://mlsanity.com/paper/2105.06499/) (2021)
- [Improved Algorithms for Neural Active Learning](https://mlsanity.com/paper/2210.00423/) (2022)
- [Understanding Uncertainty Sampling via Equivalent Loss](https://mlsanity.com/paper/2307.02719/) (2023)
- [Leveraging Importance Weights in Subset Selection](https://mlsanity.com/paper/2301.12052/) (2023)
- [What killed the Convex Booster ?](https://mlsanity.com/paper/2205.09628/) (2022)
- [On the Convergence of Loss and Uncertainty-based Active Learning Algorithms](https://mlsanity.com/paper/2312.13927/) (2023)
- [Improve Cost Efficiency of Active Learning over Noisy Dataset](https://mlsanity.com/paper/2403.01346/) (2024)
- [Nearly Optimal Active Preference Learning and Its Application to LLM Alignment](https://mlsanity.com/paper/2602.01581/) (2026)

## Abstract (arXiv)

Uncertainty sampling, a popular active learning algorithm, is used to reduce the amount of data required to learn a classifier, but it has been observed in practice to converge to different parameters depending on the initialization and sometimes to even better parameters than standard training on all the data. In this work, we give a theoretical explanation of this phenomenon, showing that uncertainty sampling on a convex loss can be interpreted as performing a preconditioned stochastic gradient step on a smoothed version of the population zero-one loss that converges to the population zero-one loss. Furthermore, uncertainty sampling moves in a descent direction and converges to stationary points of the smoothed population zero-one loss. Experiments on synthetic and real datasets support this connection.