mlsanityData to 7 Oct 2026

ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

arXiv 2306.10087 ↗First posted 16 Jun 2023cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is currently no standardized evaluation protocol for transformer-based language models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for practitioners. To tackle this challenge, we propose the ActiveGLAE benchmark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 43 of its 68 references that are in the map. First posted on arXiv 16 Jun 2023.

Builds on

43 of 68 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  8. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
31 more
  1. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  2. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  3. 2020A Survey of Deep Active Learningcited by 547
  4. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  5. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  6. 2021True Few-Shot Learning with Language Modelscited by 343
  7. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  8. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  9. 2021Active Learning by Acquiring Contrastive Examplescited by 154
  10. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  11. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  12. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  13. 2021Revisiting Uncertainty-based Query Strategies for Active Learning with Transformerscited by 59
  14. 2020Towards Robust and Reproducible Active Learning Using Neural Networkscited by 53
  15. 2023Data-centric AI: Perspectives and Challengescited by 43
  16. 2021Effective Evaluation of Deep Active Learning on Image Classification Taskscited by 37
  17. 2021On the Importance of Effectively Adapting Pretrained Language Models for Active Learningcited by 31
  18. 2018Active Learning with Partial Feedbackcited by 27
  19. 2022PT4AL: Using Self-Supervised Pretext Tasks for Active Learningcited by 27
  20. 2023Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessmentcited by 25
  21. 2022ALLSH: Active Learning Guided by Local Sensitivity and Hardnesscited by 23
  22. 2021Diversity Enhanced Active Learning with Strictly Proper Scoring Rulescited by 22
  23. 2020Active Sentence Learning by Adversarial Uncertainty Sampling in Discrete Spacecited by 17
  24. 2021A Survey on Cost Types, Interaction Schemes, and Annotator Performance Models in Selection Algorithms for Active Learning in Classificationcited by 17
  25. 2022Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approachcited by 16
  26. 2021Multi-class Text Classification using BERT-based Active Learningcited by 11
  27. 2020Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasetscited by 6
  28. 2022An Empirical Study on the Efficacy of Deep Active Learning for Image Classificationcited by 6
  29. 2022TrustAL: Trustworthy Active Learning using Knowledge Distillationcited by 5
  30. 2022Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysiscited by 4
  31. 2022To Softmax, or not to Softmax: that is the question when applying Active Learning for Transformer Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 4; 2026: 3.

14 papers.

Sort:

  1. 2024BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacousticscited by 36
  2. 2024Active Learning for Neural PDE Solverscited by 22
  3. 2024A Cross-Domain Benchmark for Active Learningcited by 11
  4. 2024Towards Deep Active Learning in Avian Bioacousticscited by 10
  5. 2024Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classificationcited by 6
  6. 2024Efficient In-Context Medical Segmentation with Meta-driven Visual Prompt Selectioncited by 4
  7. 2025Revisiting Active Learning under (Human) Label Variationcited by 4
  8. 2023Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformerscited by 3
  9. 2025Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learningcited by 2
  10. 2026Select, Label, Evaluate: Active Testing in NLPcited by 2
  11. 2025No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Successcited by 1
  12. 2026BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learningcited by 1
2 more
  1. 2025Bayesian Active Learning By Distribution Disagreementcited by 0
  2. 2026ALINC: Active Learning for Inductive Node Classification via Graph Samplingcited by 0