mlsanityData to 7 Oct 2026

The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric

arXiv 2602.13359 ↗First posted 13 Feb 2026cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Machine learning models excel with abundant annotated data, but annotation is often costly and time-intensive. Active learning (AL) aims to improve the performance-to-annotation ratio by using query methods (QMs) to iteratively select the most informative samples. While AL research focuses mainly on QM development, the evaluation of this iterative process lacks appropriate performance metrics. This work reviews eight years of AL evaluation literature and formally introduces the speed-up factor, a quantitative multi-iteration QM performance metric that indicates the fraction of samples needed to match random sampling performance. Using four datasets from diverse domains and seven QMs of various types, we empirically evaluate the speed-up factor and compare it with state-of-the-art AL performance metrics. The results confirm the assumptions underlying the speed-up factor, demonstrate its accuracy in capturing the described fraction, and reveal its superior stability across iterations.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

22 of 45 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  7. 2017Deep Bayesian Active Learning with Image Datacited by 812
  8. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  9. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  10. 2018Do Better ImageNet Models Transfer Better?cited by 560
  11. 2020Task-Aware Variational Adversarial Active Learningcited by 84
  12. 2018Practical Obstacles to Deploying Active Learningcited by 68
10 more
  1. 2021The Shape of Learning Curves: a Reviewcited by 67
  2. 2020Towards Robust and Reproducible Active Learning Using Neural Networkscited by 53
  3. 2022Bayesian Active Learning with Fully Bayesian Gaussian Processescited by 26
  4. 2022Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernelscited by 25
  5. 2023Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessmentcited by 25
  6. 2021A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexitycited by 7
  7. 2020Exemplar Guided Active Learningcited by 6
  8. 2021Efficacy of Bayesian Neural Networks in Active Learningcited by 6
  9. 2017Episode-Based Active Learning with Bayesian Neural Networkscited by 4
  10. 2019Active Learning within Constrained Environments through Imitation of an Expert Questionercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When Active Learning Falls Short: An Empirical Study on Chemical Reaction Extractioncited by 0