# ALPBench: A Benchmark for Active Learning Pipelines on Tabular Data

arXiv: https://arxiv.org/abs/2406.17322 · HTML: https://mlsanity.com/paper/2406.17322/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings](https://mlsanity.com/paper/2010.08666/) (2020)
- [OpenML-Python: an extensible Python API for OpenML](https://mlsanity.com/paper/1911.02490/) (2019)
- [A Benchmark and Comparison of Active Learning for Logistic Regression](https://mlsanity.com/paper/1611.08618/) (2016)
- [Epistemic Uncertainty Sampling](https://mlsanity.com/paper/1909.00218/) (2019)
- [Is margin all you need? An extensive empirical study of active learning on tabular data](https://mlsanity.com/paper/2210.03822/) (2022)
- [An Expanded Benchmark that Rediscovers and Affirms the Edge of Uncertainty Sampling for Active Learning in Tabular Datasets](https://mlsanity.com/paper/2306.08954/) (2023)
- [When does Active Learning Work?](https://mlsanity.com/paper/1408.1319/) (2014)
- [PyExperimenter: Easily distribute experiments and track results](https://mlsanity.com/paper/2301.06348/) (2023)

## Built on by (most cited first, 5 of 5)

- [Uncertainty Quantification for Machine Learning: One Size Does Not Fit All](https://mlsanity.com/paper/2512.12341/) (2025)
- [Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Tasks](https://mlsanity.com/paper/2505.22538/) (2025)
- [No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Success](https://mlsanity.com/paper/2506.01992/) (2025)
- [Efficient Credal Prediction through Decalibration](https://mlsanity.com/paper/2603.08495/) (2026)
- [Survey of Active Learning Hyperparameters: Insights from a Large-Scale Experimental Grid](https://mlsanity.com/paper/2506.03817/) (2025)

## Abstract (arXiv)

In settings where only a budgeted amount of labeled data can be afforded, active learning seeks to devise query strategies for selecting the most informative data points to be labeled, aiming to enhance learning algorithms' efficiency and performance. Numerous such query strategies have been proposed and compared in the active learning literature. However, the community still lacks standardized benchmarks for comparing the performance of different query strategies. This particularly holds for the combination of query strategies with different learning algorithms into active learning pipelines and examining the impact of the learning algorithm choice. To close this gap, we propose ALPBench, which facilitates the specification, execution, and performance monitoring of active learning pipelines. It has built-in measures to ensure evaluations are done reproducibly, saving exact dataset splits and hyperparameter settings of used algorithms. In total, ALPBench consists of 86 real-world tabular classification datasets and 5 active learning settings, yielding 430 active learning problems. To demonstrate its usefulness and broad compatibility with various learning algorithms and query strategies, we conduct an exemplary study evaluating 9 query strategies paired with 8 learning algorithms in 2 different settings. We provide ALPBench here: https://github.com/ValentinMargraf/ActiveLearningPipelines.