mlsanityData to 14 Oct 2026

Boosting Active Learning with Knowledge Transfer

arXiv 2509.15805 ↗First posted 19 Sep 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Uncertainty estimation is at the core of Active Learning (AL). Most existing methods resort to complex auxiliary models and advanced training fashions to estimate uncertainty for unlabeled data. These models need special design and hence are difficult to train especially for domain tasks, such as Cryo-Electron Tomography (cryo-ET) classification in computational biology. To address this challenge, we propose a novel method using knowledge transfer to boost uncertainty estimation in AL. Specifically, we exploit the teacher-student mode where the teacher is the task model in AL and the student is an auxiliary model that learns from the teacher. We train the two models simultaneously in each AL cycle and adopt a certain distance between the model outputs to measure uncertainty for unlabeled data. The student model is task-agnostic and does not rely on special training fashions (e.g. adversarial), making our method suitable for various tasks. More importantly, we demonstrate that data uncertainty is not tied to concrete value of task loss but closely related to the upper-bound of task loss. We conduct extensive experiments to validate the proposed method on classical computer vision tasks and cryo-ET challenges. The results demonstrate its efficacy and efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

30 of 55 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  9. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  10. 2017Deep Bayesian Active Learning with Image Datacited by 819
  11. 2019Learning Loss for Active Learningcited by 336
  12. 2017Dilated Residual Networkscited by 294
18 more
  1. 2019Variational Adversarial Active Learningcited by 284
  2. 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
  3. 2017Learning Active Learning from Datacited by 101
  4. 2018Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Networkcited by 74
  5. 2017Generative Adversarial Active Learningcited by 70
  6. 2020State-Relabeling Adversarial Active Learningcited by 60
  7. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 58
  8. 2018Adversarial Sampling for Active Learningcited by 51
  9. 2017Cost-Effective Active Learning for Melanoma Segmentationcited by 41
  10. 2024DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptationcited by 33
  11. 2025Visual Instance-aware Prompt Tuningcited by 29
  12. 2018Cost-Sensitive Active Learning for Intracranial Hemorrhage Detectioncited by 28
  13. 2024Transductive Active Learning: Theory and Applicationscited by 22
  14. 2015Hierarchical Subquery Evaluation for Active Learning on a Graphcited by 21
  15. 2025Visual Variational Autoencoder Prompt Tuningcited by 10
  16. 2024Adaptive Stochastic Weight Averagingcited by 4
  17. 2024Composite Active Learning: Towards Multi-Domain Active Learning with Theoretical Guaranteescited by 3
  18. 2024Deep Active Learning with Manifold-preserving Trajectory Samplingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Revisiting Active Learning in the Era of Vision Foundation Models
  2. 2025Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration
  3. 2025PCoreSet: Effective Active Learning through Knowledge Distillation from Vision-Language Models
  4. 2024Uncertainty-preserving deep knowledge tracing with state-space models
  5. 2025Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
  6. 2024Active Learning for Vision-Language Models
  7. 2025Uncertainty-Aware Dual-Student Knowledge Distillation for Efficient Image Classification
  8. 2026Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors