mlsanityData to 14 Oct 2026

ALScope: A Unified Toolkit for Deep Active Learning

arXiv 2508.04937 ↗First posted 6 Aug 2025cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep Active Learning (DAL) reduces annotation costs by selecting the most informative unlabeled samples during training. As real-world applications become more complex, challenges stemming from distribution shifts (e.g., open-set recognition) and data imbalance have gained increasing attention, prompting the development of numerous DAL algorithms. However, the lack of a unified platform has hindered fair and systematic evaluation under diverse conditions. Therefore, we present a new DAL platform ALScope for classification tasks, integrating 10 datasets from computer vision (CV) and natural language processing (NLP), and 21 representative DAL algorithms, including both classical baselines and recent approaches designed to handle challenges such as distribution shifts and data imbalance. This platform supports flexible configuration of key experimental factors, ranging from algorithm and dataset choices to task-specific factors like out-of-distribution (OOD) sample ratio, and class imbalance ratio, enabling comprehensive and realistic evaluation. We conduct extensive experiments on this platform under various settings. Our findings show that: (1) DAL algorithms' performance varies significantly across domains and task settings; (2) in non-standard scenarios such as imbalanced and open-set settings, DAL algorithms show room for improvement and require further investigation; and (3) some algorithms achieve good performance, but require significantly longer selection time.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

23 of 52 references.

References · 23

  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  2. 2017Deep Bayesian Active Learning with Image Datacited by 819
  3. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  4. 2020A Survey of Deep Active Learningcited by 554
  5. 2019Learning Loss for Active Learningcited by 336
  6. 2019Variational Adversarial Active Learningcited by 284
  7. 2019Discriminative Active Learningcited by 98
  8. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 93
  9. 2022A Comparative Survey of Deep Active Learningcited by 89
  10. 2022Active Learning for Open-set Annotationcited by 34
  11. 2023Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shiftscited by 28
  12. 2022Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learningcited by 25
11 more
  1. 2020Bayesian active learning for production, a systematic study and a reusable librarycited by 15
  2. 2022Deep Active Learning with Noise Stabilitycited by 14
  3. 2024A Cross-Domain Benchmark for Active Learningcited by 12
  4. 2022TiDAL: Learning Training Dynamics for Active Learningcited by 11
  5. 2023Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentationcited by 11
  6. 2023Monocular 3D Object Detection with LiDAR Guided Semi Supervised Active Learningcited by 10
  7. 2023Active Learning for Semantic Segmentation with Multi-class Label Querycited by 9
  8. 2023Bayesian Estimate of Mean Proper Scores for Diversity-Enhanced Active Learningcited by 6
  9. 2024Inconsistency-Based Data-Centric Active Open-Set Annotationcited by 5
  10. 2024AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasetscited by 5
  11. 2018Active Learning in Recommendation Systems with Multi-level User Preferencescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Next Generation Active Learning: Mixture of LLMs in the Loopcited by 3