mlsanityData to 14 Oct 2026

Boosting Active Learning via Improving Test Performance

arXiv 2112.05683 ↗First posted 10 Dec 2021cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Central to active learning (AL) is what data should be selected for annotation. Existing works attempt to select highly uncertain or informative data for annotation. Nevertheless, it remains unclear how selected data impacts the test performance of the task model used in AL. In this work, we explore such an impact by theoretically proving that selecting unlabeled data of higher gradient norm leads to a lower upper-bound of test loss, resulting in better test performance. However, due to the lack of label information, directly computing gradient norm for unlabeled data is infeasible. To address this challenge, we propose two schemes, namely expected-gradnorm and entropy-gradnorm. The former computes the gradient norm by constructing an expected empirical loss while the latter constructs an unsupervised loss with entropy. Furthermore, we integrate the two schemes in a universal AL framework. We evaluate our method on classical image classification and semantic segmentation tasks. To demonstrate its competency in domain applications and its robustness to noise, we also validate our method on a cellular imaging analysis task, namely cryo-Electron Tomography subtomogram classification. Results demonstrate that our method achieves superior performance against the state of the art. Our source code is available at https://github.com/xulabs/aitom/blob/master/doc/projects/al_gradnorm.md.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 10 Dec 2021.

Builds on

29 of 54 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  8. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  10. 2017Deep Bayesian Active Learning with Image Datacited by 819
  11. 2019Learning Loss for Active Learningcited by 336
  12. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
17 more
  1. 2017Dilated Residual Networkscited by 294
  2. 2019Variational Adversarial Active Learningcited by 284
  3. 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 158
  4. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  5. 2017Learning Active Learning from Datacited by 101
  6. 2021Influence Selection for Active Learningcited by 84
  7. 2018Knowledge Transfer with Jacobian Matchingcited by 76
  8. 2018Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Networkcited by 74
  9. 2017Generative Adversarial Active Learningcited by 70
  10. 2020Sequential Graph Convolutional Network for Active Learningcited by 69
  11. 2020State-Relabeling Adversarial Active Learningcited by 60
  12. 2018Adversarial Sampling for Active Learningcited by 51
  13. 2017Cost-Effective Active Learning for Melanoma Segmentationcited by 41
  14. 2018Cost-Sensitive Active Learning for Intracranial Hemorrhage Detectioncited by 28
  15. 2015Hierarchical Subquery Evaluation for Active Learning on a Graphcited by 21
  16. 2019Deep Active Learning with Adaptive Acquisitioncited by 17
  17. 2016Piecewise convexity of artificial neural networkscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 7; 2025: 0; 2026: 1.

19 papers.

Sort:

  1. 2022A Comparative Survey of Deep Active Learningcited by 89
  2. 2024A Survey on Deep Active Learning: Recent Advances and New Frontierscited by 54
  3. 2023A comprehensive survey on deep active learning in medical image analysiscited by 35
  4. 2023Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shiftscited by 28
  5. 2022Deep Active Learning with Noise Stabilitycited by 14
  6. 2023Controllable Textual Inversion for Personalized Text-to-Image Generationcited by 14
  7. 2024Fairness Without Harm: An Influence-Guided Active Sampling Approachcited by 12
  8. 2024Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentationcited by 8
  9. 2022Deep Active Learning for Computer Vision: Past and Futurecited by 7
  10. 2024Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Modelcited by 6
  11. 2023Deep Active Learning in the Presence of Label Noise: A Surveycited by 5
  12. 2024Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrievalcited by 5
7 more
  1. 2023Confidence-based federated distillation for vision-based lane-centeringcited by 3
  2. 2023SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentationcited by 2
  3. 2023How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learningcited by 2
  4. 2024Deep Active Learning with Manifold-preserving Trajectory Samplingcited by 2
  5. 2024LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Trainingcited by 2
  6. 2023Active Learning Guided by Efficient Surrogate Learnerscited by 1
  7. 2026Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimizationcited by 0