# Testing the Assumptions of Active Learning for Translation Tasks with Few Samples

arXiv: https://arxiv.org/abs/2604.08977 · HTML: https://mlsanity.com/paper/2604.08977/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 18 of 18)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study](https://mlsanity.com/paper/1808.05697/) (2018)
- [Sampling Bias in Deep Active Classification: An Empirical Study](https://mlsanity.com/paper/1909.09389/) (2019)
- [Self-Paced Learning for Neural Machine Translation](https://mlsanity.com/paper/2010.04505/) (2020)
- [On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study](https://mlsanity.com/paper/2304.08653/) (2023)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [COMET-QE and Active Learning for Low-Resource Machine Translation](https://mlsanity.com/paper/2210.15696/) (2022)
- [Combining Data Generation and Active Learning for Low-Resource Question Answering](https://mlsanity.com/paper/2211.14880/) (2022)
- [Improving the Calibration of Confidence Scores in Text Generation Using the Output Distribution's Characteristics](https://mlsanity.com/paper/2506.00637/) (2025)

## Abstract (arXiv)

Active learning (AL) is a training paradigm for selecting unlabeled samples for annotation to improve model performance on a test set, which is useful when only a limited number of samples can be annotated. These algorithms often work by optimizing for the informativeness and diversity of the training data to be annotated. Recent work found that AL strategies fail to outperform random sampling on various language generation tasks when using 100-500 samples. To understand AL's poor performance when only using few samples, we investigate whether the core assumptions underlying AL strategies hold. We find that neither the informativeness nor diversity of the training data, which AL strategies optimize for, are correlated with test set performance. Instead, factors like the ordering of the training samples and interactions with pre-training data have a larger impact on performance. This suggests that future AL methods must take these factors into account in order to work with very few samples.