Active Learning from Imperfect Labelers
Not on the map. Why?
Abstract (arXiv)
We study active learning where the labeler can not only return incorrect labels but also abstain from labeling. We consider different noise and abstention conditions of the labeler. We propose an algorithm which utilizes abstention responses, and analyze its statistical consistency and query complexity under fairly natural assumptions on the noise and abstention rate of the labeler. This algorithm is adaptive in a sense that it can automatically request less queries with a more informed or less noisy labeler. We couple our algorithm with lower bounds to show that under some technical conditions, it achieves nearly optimal query complexity.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 1; 2023: 6; 2024: 1; 2025: 1; 2026: 1.
13 papers.
Sort:
- 2021Exponential Savings in Agnostic Active Learning through Abstentioncited by 23
- 2021Backward-Compatible Prediction Updates: A Probabilistic Approachcited by 14
- 2021Nuances in Margin Conditions Determine Gains in Active Learningcited by 9
- 2023Reliable learning in challenging environmentscited by 9
- 2023Deep Active Learning in the Presence of Label Noise: A Surveycited by 5
- 2022Improved Adaptive Algorithm for Scalable Active Learning with Weak Labelercited by 4
- 2023Online Decision Mediationcited by 4
- 2025Revisiting Active Learning under (Human) Label Variationcited by 4
- 2023Deep Active Learning with Noisy Oracle in Object Detectioncited by 2
- 2024GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noisecited by 2
- 2023Exploiting Counter-Examples for Active Learning with Partial labelscited by 1
- 2023Active learning with biased non-response to label requestscited by 0
- 2026An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations25 Apr 2026
- 2025Revisiting Active Learning under (Human) Label Variation3 Jul 2025
- 2024GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise8 Nov 2024
- 2023Active learning with biased non-response to label requests13 Dec 2023
- 2023Online Decision Mediation28 Oct 2023
- 2023Deep Active Learning with Noisy Oracle in Object Detection30 Sep 2023
- 2023Exploiting Counter-Examples for Active Learning with Partial labels14 Jul 2023
- 2023Reliable learning in challenging environments6 Apr 2023
- 2023Deep Active Learning in the Presence of Label Noise: A Survey22 Feb 2023
- 2022Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler4 Nov 2022
- 2021Nuances in Margin Conditions Determine Gains in Active Learning16 Oct 2021
- 2021Backward-Compatible Prediction Updates: A Probabilistic Approach2 Jul 2021
1 more
- 2021Exponential Savings in Agnostic Active Learning through Abstentioncited by 23
- 2021Backward-Compatible Prediction Updates: A Probabilistic Approachcited by 14
- 2021Nuances in Margin Conditions Determine Gains in Active Learningcited by 9
- 2023Reliable learning in challenging environmentscited by 9
- 2023Deep Active Learning in the Presence of Label Noise: A Surveycited by 5
- 2022Improved Adaptive Algorithm for Scalable Active Learning with Weak Labelercited by 4
- 2023Online Decision Mediationcited by 4
- 2025Revisiting Active Learning under (Human) Label Variationcited by 4
- 2023Deep Active Learning with Noisy Oracle in Object Detectioncited by 2
- 2024GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noisecited by 2
- 2023Exploiting Counter-Examples for Active Learning with Partial labelscited by 1
- 2023Active learning with biased non-response to label requestscited by 0
12 most cited of 13 All 13 (JSON)
- 2026An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations25 Apr 2026
- 2025Revisiting Active Learning under (Human) Label Variation3 Jul 2025
- 2024GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise8 Nov 2024
- 2023Active learning with biased non-response to label requests13 Dec 2023
- 2023Online Decision Mediation28 Oct 2023
- 2023Deep Active Learning with Noisy Oracle in Object Detection30 Sep 2023
- 2023Exploiting Counter-Examples for Active Learning with Partial labels14 Jul 2023
- 2023Reliable learning in challenging environments6 Apr 2023
- 2023Deep Active Learning in the Presence of Label Noise: A Survey22 Feb 2023
- 2022Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler4 Nov 2022
- 2021Nuances in Margin Conditions Determine Gains in Active Learning16 Oct 2021
- 2021Backward-Compatible Prediction Updates: A Probabilistic Approach2 Jul 2021
12 newest of 13 All 13 (JSON)