mlsanityData to 14 Oct 2026

BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning

arXiv 2603.13109 ↗First posted 13 Mar 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Active learning (AL) aims to reduce annotation costs while maximizing model performance by iteratively selecting valuable instances. While foundation models have made it easier to identify these instances, existing selection strategies still lack robustness across different models, annotation budgets, and datasets. To highlight the potential weaknesses of existing AL strategies and provide a reference point for research, we explore oracle strategies, i.e., strategies that approximate the optimal selection by accessing ground-truth information unavailable in practical AL scenarios. Current oracle strategies, however, fail to scale effectively to large datasets and complex deep neural networks. To tackle these limitations, we introduce the Best-of-Strategy Selector (BoSS), a scalable oracle strategy designed for large-scale AL scenarios. BoSS constructs a set of candidate batches through an ensemble of selection strategies and then selects the batch yielding the highest performance gain. As an ensemble of selection strategies, BoSS can be easily extended with new state-of-the-art strategies as they emerge, ensuring it remains a reliable oracle strategy in the future. Our evaluation demonstrates that i) BoSS outperforms existing oracle strategies, ii) state-of-the-art AL strategies still fall noticeably short of oracle performance, especially in large-scale datasets with many classes, and iii) one possible solution to counteract the inconsistent performance of AL strategies might be to employ an ensemble-based approach for the selection.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

28 of 47 references.

References · 28

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  10. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  11. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  12. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
16 more
  1. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
  2. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 133
  3. 2021Gone Fishing: Neural Active Learning with Fisher Embeddingscited by 88
  4. 2022Active Learning by Feature Mixingcited by 63
  5. 2020Towards Robust and Reproducible Active Learning Using Neural Networkscited by 54
  6. 2021Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learningcited by 36
  7. 2023Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessmentcited by 26
  8. 2023How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budgetcited by 17
  9. 2023ActiveGLAE: A Benchmark for Deep Active Learning with Transformerscited by 14
  10. 2024Revisiting Active Learning in the Era of Vision Foundation Modelscited by 14
  11. 2024A Cross-Domain Benchmark for Active Learningcited by 12
  12. 2024Towards Deep Active Learning in Avian Bioacousticscited by 10
  13. 2020Toward Optimal Probabilistic Active Learning Using a Bayesian Approachcited by 6
  14. 2024Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classificationcited by 6
  15. 2024dopanim: A Dataset of Doppelganger Animals with Noisy Annotations from Multiple Humanscited by 3
  16. 2025Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026BioDCASE: Active Learning for Bioacousticscited by 1
  2. 2026One Loop, Two Gains: Can Active Learning win the Lottery for Free?cited by 0