# The Speed-up Factor: A Quantitative Multi-Iteration Active Learning Performance Metric

arXiv: https://arxiv.org/abs/2602.13359 · HTML: https://mlsanity.com/paper/2602.13359/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Task-Aware Variational Adversarial Active Learning](https://mlsanity.com/paper/2002.04709/) (2020)
- [The Shape of Learning Curves: a Review](https://mlsanity.com/paper/2103.10948/) (2021)
- [Practical Obstacles to Deploying Active Learning](https://mlsanity.com/paper/1807.04801/) (2018)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Bayesian Active Learning with Fully Bayesian Gaussian Processes](https://mlsanity.com/paper/2205.10186/) (2022)
- [Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment](https://mlsanity.com/paper/2301.10625/) (2023)
- [Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels](https://mlsanity.com/paper/2206.12569/) (2022)
- [A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexity](https://mlsanity.com/paper/2104.03525/) (2021)
- [Exemplar Guided Active Learning](https://mlsanity.com/paper/2011.01285/) (2020)
- [Efficacy of Bayesian Neural Networks in Active Learning](https://mlsanity.com/paper/2104.00896/) (2021)
- [Episode-Based Active Learning with Bayesian Neural Networks](https://mlsanity.com/paper/1703.07473/) (2017)
- [Active Learning within Constrained Environments through Imitation of an Expert Questioner](https://mlsanity.com/paper/1907.00921/) (2019)

## Built on by (most cited first, 1 of 1)

- [When Active Learning Falls Short: An Empirical Study on Chemical Reaction Extraction](https://mlsanity.com/paper/2604.19335/) (2026)

## Abstract (arXiv)

Machine learning models excel with abundant annotated data, but annotation is often costly and time-intensive. Active learning (AL) aims to improve the performance-to-annotation ratio by using query methods (QMs) to iteratively select the most informative samples. While AL research focuses mainly on QM development, the evaluation of this iterative process lacks appropriate performance metrics. This work reviews eight years of AL evaluation literature and formally introduces the speed-up factor, a quantitative multi-iteration QM performance metric that indicates the fraction of samples needed to match random sampling performance. Using four datasets from diverse domains and seven QMs of various types, we empirically evaluate the speed-up factor and compare it with state-of-the-art AL performance metrics. The results confirm the assumptions underlying the speed-up factor, demonstrate its accuracy in capturing the described fraction, and reveal its superior stability across iterations.