# An Expanded Benchmark that Rediscovers and Affirms the Edge of Uncertainty Sampling for Active Learning in Tabular Datasets

arXiv: https://arxiv.org/abs/2306.08954 · HTML: https://mlsanity.com/paper/2306.08954/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 85 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Learning Active Learning from Data](https://mlsanity.com/paper/1703.03365/) (2017)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [Multiple instance active learning for object detection](https://mlsanity.com/paper/2104.02324/) (2021)
- [Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering](https://mlsanity.com/paper/2107.02331/) (2021)
- [Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers](https://mlsanity.com/paper/2107.05687/) (2021)
- [ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Prediction](https://mlsanity.com/paper/2007.03196/) (2020)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Active Learning for Deep Object Detection](https://mlsanity.com/paper/1809.09875/) (2018)
- [Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation](https://mlsanity.com/paper/2101.02323/) (2021)
- [A Survey of Active Learning for Natural Language Processing](https://mlsanity.com/paper/2210.10109/) (2022)
- [Effective Evaluation of Deep Active Learning on Image Classification Tasks](https://mlsanity.com/paper/2106.15324/) (2021)
- [A Benchmark and Comparison of Active Learning for Logistic Regression](https://mlsanity.com/paper/1611.08618/) (2016)
- [modAL: A modular active learning framework for Python](https://mlsanity.com/paper/1805.00979/) (2018)
- [Fair Active Learning](https://mlsanity.com/paper/2001.01796/) (2020)
- [Active Learning for Regression Using Greedy Sampling](https://mlsanity.com/paper/1808.04245/) (2018)
- [libact: Pool-based Active Learning in Python](https://mlsanity.com/paper/1710.00379/) (2017)
- [On the Relationship between Data Efficiency and Error for Uncertainty Sampling](https://mlsanity.com/paper/1806.06123/) (2018)
- [ALiPy: Active Learning in Python](https://mlsanity.com/paper/1901.03802/) (2019)
- [Is margin all you need? An extensive empirical study of active learning on tabular data](https://mlsanity.com/paper/2210.03822/) (2022)
- [DECAL: DEployable Clinical Active Learning](https://mlsanity.com/paper/2206.10120/) (2022)
- [Deep Active Learning for Text Classification with Diverse Interpretations](https://mlsanity.com/paper/2108.10687/) (2021)

## Built on by (most cited first, 4 of 4)

- [ALPBench: A Benchmark for Active Learning Pipelines on Tabular Data](https://mlsanity.com/paper/2406.17322/) (2024)
- [On the Query Strategies for Efficient Online Active Distillation](https://mlsanity.com/paper/2309.01612/) (2023)
- [regAL: Python Package for Active Learning of Regression Problems](https://mlsanity.com/paper/2410.17917/) (2024)
- [Survey of Active Learning Hyperparameters: Insights from a Large-Scale Experimental Grid](https://mlsanity.com/paper/2506.03817/) (2025)

## Abstract (arXiv)

Active Learning (AL) addresses the crucial challenge of enabling machines to efficiently gather labeled examples through strategic queries. Among the many AL strategies, Uncertainty Sampling (US) stands out as one of the most widely adopted. US queries the example(s) that the current model finds uncertain, proving to be both straightforward and effective. Despite claims in the literature suggesting superior alternatives to US, community-wide acceptance remains elusive. In fact, existing benchmarks for tabular datasets present conflicting conclusions on the continued competitiveness of US. In this study, we review the literature on AL strategies in the last decade and build the most comprehensive open-source AL benchmark to date to understand the relative merits of different AL strategies. The benchmark surpasses existing ones by encompassing a broader coverage of strategies, models, and data. Through our investigation of the conflicting conclusions in existing tabular AL benchmarks by evaluation under broad AL experimental settings, we uncover fresh insights into the often-overlooked issue of using machine learning models--**model compatibility** in the context of US. Specifically, we notice that adopting the different models for the querying unlabeled examples and learning tasks would degrade US's effectiveness. Notably, our findings affirm that US maintains a competitive edge over other strategies when paired with compatible models. These findings have practical implications and provide a concrete recipe for AL practitioners, empowering them to make informed decisions when working with tabular classifications with limited labeled data. The code for this project is available on https://github.com/ariapoy/active-learning-benchmark.