# Active Learning from Imperfect Labelers

arXiv: https://arxiv.org/abs/1610.09730 · HTML: https://mlsanity.com/paper/1610.09730/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2016.

## Built on by (most cited first, 13 of 13)

- [Exponential Savings in Agnostic Active Learning through Abstention](https://mlsanity.com/paper/2102.00451/) (2021)
- [Backward-Compatible Prediction Updates: A Probabilistic Approach](https://mlsanity.com/paper/2107.01057/) (2021)
- [Reliable learning in challenging environments](https://mlsanity.com/paper/2304.03370/) (2023)
- [Nuances in Margin Conditions Determine Gains in Active Learning](https://mlsanity.com/paper/2110.08418/) (2021)
- [Deep Active Learning in the Presence of Label Noise: A Survey](https://mlsanity.com/paper/2302.11075/) (2023)
- [Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler](https://mlsanity.com/paper/2211.02233/) (2022)
- [Online Decision Mediation](https://mlsanity.com/paper/2310.18601/) (2023)
- [Revisiting Active Learning under (Human) Label Variation](https://mlsanity.com/paper/2507.02593/) (2025)
- [Deep Active Learning with Noisy Oracle in Object Detection](https://mlsanity.com/paper/2310.00372/) (2023)
- [GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise](https://mlsanity.com/paper/2411.05939/) (2024)
- [Exploiting Counter-Examples for Active Learning with Partial labels](https://mlsanity.com/paper/2307.07413/) (2023)
- [Active learning with biased non-response to label requests](https://mlsanity.com/paper/2312.08150/) (2023)
- [An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations](https://mlsanity.com/paper/2604.23290/) (2026)

## Abstract (arXiv)

We study active learning where the labeler can not only return incorrect labels but also abstain from labeling. We consider different noise and abstention conditions of the labeler. We propose an algorithm which utilizes abstention responses, and analyze its statistical consistency and query complexity under fairly natural assumptions on the noise and abstention rate of the labeler. This algorithm is adaptive in a sense that it can automatically request less queries with a more informed or less noisy labeler. We couple our algorithm with lower bounds to show that under some technical conditions, it achieves nearly optimal query complexity.