mlsanityData to 7 Oct 2026

On the Limitations of Simulating Active Learning

arXiv 2305.13342 ↗First posted 21 May 2023cs.LG cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation, aiming to improve over random sampling. However, performing AL experiments with human annotations on-the-fly is a laborious and expensive process, thus unrealistic for academic research. An easy fix to this impediment is to simulate AL, by treating an already labeled and publicly available dataset as the pool of unlabeled data. In this position paper, we first survey recent literature and highlight the challenges across all different steps within the AL loop. We further unveil neglected caveats in the experimental setup that can significantly affect the quality of AL research. We continue with an exploration of how the simulation setting can govern empirical findings, arguing that it might be one of the answers behind the ever posed question ``why do active learning algorithms sometimes fail to outperform random sampling?''. We argue that evaluating AL algorithms on available labeled datasets might provide a lower bound as to their effectiveness in real data. We believe it is essential to collectively shape the best practices for AL research, particularly as engineering advancements in LLMs push the research focus towards data-driven approaches (e.g., data efficiency, alignment, fairness). In light of this, we have developed guidelines for future work. Our aim is to draw attention to these limitations within the community, in the hope of finding ways to address them.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 69 of its 120 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

69 of 120 references.

References · 69

  1. 2017Deep reinforcement learning from human preferencescited by 3,846
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  4. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  5. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  6. 2017Deep Bayesian Active Learning with Image Datacited by 812
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  8. 2018Deep Visual Domain Adaptation: A Surveycited by 634
  9. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  10. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  11. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  12. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
38 more
  1. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  2. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  3. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  4. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  5. 2022Active Example Selection for In-Context Learningcited by 195
  6. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 164
  7. 2019A Meta-Transfer Objective for Learning to Disentangle Causal Mechanismscited by 158
  8. 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 155
  9. 2021Active Learning by Acquiring Contrastive Examplescited by 154
  10. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  11. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  12. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  13. 2017Deep Active Learning for Named Entity Recognitioncited by 108
  14. 2017Learning Active Learning from Datacited by 100
  15. 2019Discriminative Active Learningcited by 97
  16. 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Studycited by 90
  17. 2019Diverse mini-batch Active Learningcited by 88
  18. 2017Deep Active Learning over the Long Tailcited by 82
  19. 2020We Need to Talk About Random Splitscited by 70
  20. 2018Practical Obstacles to Deploying Active Learningcited by 68
  21. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  22. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 59
  23. 2020SeqMix: Augmenting Active Sequence Labeling via Sequence Mixupcited by 52
  24. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  25. 2020A Survey of Active Learning for Text Classification using Deep Neural Networkscited by 50
  26. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  27. 2021Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimatescited by 39
  28. 2019Low-resource Deep Entity Resolution with Transfer and Active Learningcited by 37
  29. 2020A Call for More Rigor in Unsupervised Cross-lingual Learningcited by 37
  30. 2019Sampling Bias in Deep Active Classification: An Empirical Studycited by 36
  31. 2022Active Learning Helps Pretrained Models Learn the Intended Taskcited by 33
  32. 2021On the Importance of Effectively Adapting Pretrained Language Models for Active Learningcited by 31
  33. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  34. 2017Learning Algorithms for Active Learningcited by 30
  35. 2021Cartography Active Learningcited by 26
  36. 2021Analyzing Dynamic Adversarial Training Data in the Limitcited by 24
  37. 2022ALLSH: Active Learning Guided by Local Sensitivity and Hardnesscited by 23
  38. 2021Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outlierscited by 22

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5; 2025: 2.

9 papers.

  1. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  2. 2024Enhancing Text Classification through LLM-Driven Active Learning and Human Annotationcited by 20
  3. 2024Annotator-Centric Active Learning for Subjective NLP Taskscited by 18
  4. 2024On the Pros and Cons of Active Learning for Moral Preference Elicitationcited by 6
  5. 2024AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasetscited by 4
  6. 2023Active Learning for Natural Language Generationcited by 3
  7. 2024On the Fragility of Active Learners for Text Classificationcited by 2
  8. 2025Reassessing Active Learning Adoption in Contemporary NLP: A Community Surveycited by 2
  9. 2025Op-Fed: Opinion, Stance, and Monetary Policy Annotations on FOMC Transcripts Using Active Learningcited by 1