# ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

arXiv: https://arxiv.org/abs/2306.10087 · HTML: https://mlsanity.com/paper/2306.10087/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 68 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 30 of 43)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Active Learning by Acquiring Contrastive Examples](https://mlsanity.com/paper/2109.03764/) (2021)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers](https://mlsanity.com/paper/2107.05687/) (2021)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Data-centric AI: Perspectives and Challenges](https://mlsanity.com/paper/2301.04819/) (2023)
- [Effective Evaluation of Deep Active Learning on Image Classification Tasks](https://mlsanity.com/paper/2106.15324/) (2021)
- [On the Importance of Effectively Adapting Pretrained Language Models for Active Learning](https://mlsanity.com/paper/2104.08320/) (2021)
- [Active Learning with Partial Feedback](https://mlsanity.com/paper/1802.07427/) (2018)

## Built on by (most cited first, 14 of 14)

- [BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics](https://mlsanity.com/paper/2403.10380/) (2024)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [A Cross-Domain Benchmark for Active Learning](https://mlsanity.com/paper/2408.00426/) (2024)
- [Towards Deep Active Learning in Avian Bioacoustics](https://mlsanity.com/paper/2406.18621/) (2024)
- [Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification](https://mlsanity.com/paper/2404.08981/) (2024)
- [Efficient In-Context Medical Segmentation with Meta-driven Visual Prompt Selection](https://mlsanity.com/paper/2407.11188/) (2024)
- [Revisiting Active Learning under (Human) Label Variation](https://mlsanity.com/paper/2507.02593/) (2025)
- [Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformers](https://mlsanity.com/paper/2308.07121/) (2023)
- [Select, Label, Evaluate: Active Testing in NLP](https://mlsanity.com/paper/2603.21840/) (2026)
- [Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning](https://mlsanity.com/paper/2511.22344/) (2025)
- [BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning](https://mlsanity.com/paper/2603.13109/) (2026)
- [No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Success](https://mlsanity.com/paper/2506.01992/) (2025)
- [Bayesian Active Learning By Distribution Disagreement](https://mlsanity.com/paper/2501.01248/) (2025)
- [ALINC: Active Learning for Inductive Node Classification via Graph Sampling](https://mlsanity.com/paper/2606.04647/) (2026)

## Abstract (arXiv)

Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is currently no standardized evaluation protocol for transformer-based language models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for practitioners. To tackle this challenge, we propose the ActiveGLAE benchmark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.