# Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey

arXiv: https://arxiv.org/abs/2503.09701 · HTML: https://mlsanity.com/paper/2503.09701/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 82 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 30 of 32)

- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Active Learning by Acquiring Contrastive Examples](https://mlsanity.com/paper/2109.03764/) (2021)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Explainable Deep Learning: A Field Guide for the Uninitiated](https://mlsanity.com/paper/2004.14545/) (2020)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [LLMaAA: Making Large Language Models as Active Annotators](https://mlsanity.com/paper/2310.19596/) (2023)
- [Practical Obstacles to Deploying Active Learning](https://mlsanity.com/paper/1807.04801/) (2018)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers](https://mlsanity.com/paper/2107.05687/) (2021)
- [Active Learning Principles for In-Context Learning with Large Language Models](https://mlsanity.com/paper/2305.14264/) (2023)
- [A Survey of Active Learning for Text Classification using Deep Neural Networks](https://mlsanity.com/paper/2008.07267/) (2020)
- [Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates](https://mlsanity.com/paper/2101.08133/) (2021)
- [On the Importance of Effectively Adapting Pretrained Language Models for Active Learning](https://mlsanity.com/paper/2104.08320/) (2021)
- [Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications](https://mlsanity.com/paper/2205.06828/) (2022)
- [TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics](https://mlsanity.com/paper/2101.10273/) (2021)
- [What Do NLP Researchers Believe? Results of the NLP Community Metasurvey](https://mlsanity.com/paper/2208.12852/) (2022)
- [It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance](https://mlsanity.com/paper/2305.09022/) (2023)
- [What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects](https://mlsanity.com/paper/2402.11968/) (2024)
- [Position: Key Claims in LLM Research Have a Long Tail of Footnotes](https://mlsanity.com/paper/2308.07120/) (2023)
- [Annotator-Centric Active Learning for Subjective NLP Tasks](https://mlsanity.com/paper/2404.15720/) (2024)
- [ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios](https://mlsanity.com/paper/2405.10808/) (2024)
- [ACTOR: Active Learning with Annotator-specific Classification Heads to Embrace Human Label Variation](https://mlsanity.com/paper/2310.14979/) (2023)
- [On the Limitations of Simulating Active Learning](https://mlsanity.com/paper/2305.13342/) (2023)
- [ActiveAED: A Human in the Loop Improves Annotation Error Detection](https://mlsanity.com/paper/2305.20045/) (2023)
- [Multilingual Detection of Personal Employment Status on Twitter](https://mlsanity.com/paper/2203.09178/) (2022)
- [Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours](https://mlsanity.com/paper/2208.01483/) (2022)
- [Onception: Active Learning with Expert Advice for Real World Machine Translation](https://mlsanity.com/paper/2203.04507/) (2022)
- [On Dataset Transferability in Active Learning for Transformers](https://mlsanity.com/paper/2305.09807/) (2023)
- [EvolveMT: an Ensemble MT Engine Improving Itself with Usage Only](https://mlsanity.com/paper/2306.11823/) (2023)
- [On the Fragility of Active Learners for Text Classification](https://mlsanity.com/paper/2403.15744/) (2024)

## Built on by (most cited first, 2 of 2)

- [Survey of Active Learning Hyperparameters: Insights from a Large-Scale Experimental Grid](https://mlsanity.com/paper/2506.03817/) (2025)
- [Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection](https://mlsanity.com/paper/2604.13899/) (2026)

## Abstract (arXiv)

Supervised learning relies on data annotation which usually is time-consuming and therefore expensive. A longstanding strategy to reduce annotation costs is active learning, an iterative process, in which a human annotates only data instances deemed informative by a model. Research in active learning has made considerable progress, especially with the rise of large language models (LLMs). However, we still know little about how these remarkable advances have translated into real-world applications, or contributed to removing key barriers to active learning adoption. To fill in this gap, we conduct an online survey in the NLP community to collect previously intangible insights on current implementation practices, common obstacles in application, and future prospects in active learning. We also reassess the perceived relevance of data annotation and active learning as fundamental assumptions. Our findings show that data annotation is expected to remain important and active learning to stay relevant while benefiting from LLMs. Consistent with a community survey from over 15 years ago, three key challenges yet persist -- setup complexity, uncertain cost reduction, and tooling -- for which we propose alleviation strategies. We publish an anonymized version of the dataset.