# ALScope: A Unified Toolkit for Deep Active Learning

arXiv: https://arxiv.org/abs/2508.04937 · HTML: https://mlsanity.com/paper/2508.04937/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 23 of 23)

- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Discriminative Active Learning](https://mlsanity.com/paper/1907.06347/) (2019)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [Active Learning for Open-set Annotation](https://mlsanity.com/paper/2201.06758/) (2022)
- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning](https://mlsanity.com/paper/2210.07805/) (2022)
- [Bayesian active learning for production, a systematic study and a reusable library](https://mlsanity.com/paper/2006.09916/) (2020)
- [Deep Active Learning with Noise Stability](https://mlsanity.com/paper/2205.13340/) (2022)
- [A Cross-Domain Benchmark for Active Learning](https://mlsanity.com/paper/2408.00426/) (2024)
- [TiDAL: Learning Training Dynamics for Active Learning](https://mlsanity.com/paper/2210.06788/) (2022)
- [Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2310.20293/) (2023)
- [Monocular 3D Object Detection with LiDAR Guided Semi Supervised Active Learning](https://mlsanity.com/paper/2307.08415/) (2023)
- [Active Learning for Semantic Segmentation with Multi-class Label Query](https://mlsanity.com/paper/2309.09319/) (2023)
- [Bayesian Estimate of Mean Proper Scores for Diversity-Enhanced Active Learning](https://mlsanity.com/paper/2312.10116/) (2023)
- [Inconsistency-Based Data-Centric Active Open-Set Annotation](https://mlsanity.com/paper/2401.04923/) (2024)
- [AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets](https://mlsanity.com/paper/2404.05623/) (2024)
- [Active Learning in Recommendation Systems with Multi-level User Preferences](https://mlsanity.com/paper/1811.12591/) (2018)

## Built on by (most cited first, 1 of 1)

- [Next Generation Active Learning: Mixture of LLMs in the Loop](https://mlsanity.com/paper/2601.15773/) (2026)

## Abstract (arXiv)

Deep Active Learning (DAL) reduces annotation costs by selecting the most informative unlabeled samples during training. As real-world applications become more complex, challenges stemming from distribution shifts (e.g., open-set recognition) and data imbalance have gained increasing attention, prompting the development of numerous DAL algorithms. However, the lack of a unified platform has hindered fair and systematic evaluation under diverse conditions. Therefore, we present a new DAL platform ALScope for classification tasks, integrating 10 datasets from computer vision (CV) and natural language processing (NLP), and 21 representative DAL algorithms, including both classical baselines and recent approaches designed to handle challenges such as distribution shifts and data imbalance. This platform supports flexible configuration of key experimental factors, ranging from algorithm and dataset choices to task-specific factors like out-of-distribution (OOD) sample ratio, and class imbalance ratio, enabling comprehensive and realistic evaluation. We conduct extensive experiments on this platform under various settings. Our findings show that: (1) DAL algorithms' performance varies significantly across domains and task settings; (2) in non-standard scenarios such as imbalanced and open-set settings, DAL algorithms show room for improvement and require further investigation; and (3) some algorithms achieve good performance, but require significantly longer selection time.