# BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning

arXiv: https://arxiv.org/abs/2603.13109 · HTML: https://mlsanity.com/paper/2603.13109/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Selection via Proxy: Efficient Data Selection for Deep Learning](https://mlsanity.com/paper/1906.11829/) (2019)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [Active Learning by Feature Mixing](https://mlsanity.com/paper/2203.07034/) (2022)
- [Towards Robust and Reproducible Active Learning Using Neural Networks](https://mlsanity.com/paper/2002.09564/) (2020)
- [Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning](https://mlsanity.com/paper/2106.12059/) (2021)
- [Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment](https://mlsanity.com/paper/2301.10625/) (2023)
- [How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget](https://mlsanity.com/paper/2306.03543/) (2023)
- [ActiveGLAE: A Benchmark for Deep Active Learning with Transformers](https://mlsanity.com/paper/2306.10087/) (2023)
- [Revisiting Active Learning in the Era of Vision Foundation Models](https://mlsanity.com/paper/2401.14555/) (2024)
- [A Cross-Domain Benchmark for Active Learning](https://mlsanity.com/paper/2408.00426/) (2024)
- [Towards Deep Active Learning in Avian Bioacoustics](https://mlsanity.com/paper/2406.18621/) (2024)
- [Toward Optimal Probabilistic Active Learning Using a Bayesian Approach](https://mlsanity.com/paper/2006.01732/) (2020)
- [Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification](https://mlsanity.com/paper/2404.08981/) (2024)
- [dopanim: A Dataset of Doppelganger Animals with Noisy Annotations from Multiple Humans](https://mlsanity.com/paper/2407.20950/) (2024)
- [Cleaning the Pool: Progressive Filtering of Unlabeled Pools in Deep Active Learning](https://mlsanity.com/paper/2511.22344/) (2025)

## Built on by (most cited first, 2 of 2)

- [BioDCASE: Active Learning for Bioacoustics](https://mlsanity.com/paper/2609.15255/) (2026)
- [One Loop, Two Gains: Can Active Learning win the Lottery for Free?](https://mlsanity.com/paper/2609.10311/) (2026)

## Abstract (arXiv)

Active learning (AL) aims to reduce annotation costs while maximizing model performance by iteratively selecting valuable instances. While foundation models have made it easier to identify these instances, existing selection strategies still lack robustness across different models, annotation budgets, and datasets. To highlight the potential weaknesses of existing AL strategies and provide a reference point for research, we explore oracle strategies, i.e., strategies that approximate the optimal selection by accessing ground-truth information unavailable in practical AL scenarios. Current oracle strategies, however, fail to scale effectively to large datasets and complex deep neural networks. To tackle these limitations, we introduce the Best-of-Strategy Selector (BoSS), a scalable oracle strategy designed for large-scale AL scenarios. BoSS constructs a set of candidate batches through an ensemble of selection strategies and then selects the batch yielding the highest performance gain. As an ensemble of selection strategies, BoSS can be easily extended with new state-of-the-art strategies as they emerge, ensuring it remains a reliable oracle strategy in the future. Our evaluation demonstrates that i) BoSS outperforms existing oracle strategies, ii) state-of-the-art AL strategies still fall noticeably short of oracle performance, especially in large-scale datasets with many classes, and iii) one possible solution to counteract the inconsistent performance of AL strategies might be to employ an ensemble-based approach for the selection.