# Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approach

arXiv: https://arxiv.org/abs/2106.02968 · HTML: https://mlsanity.com/paper/2106.02968/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 52 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [Deep Active Learning: Unified and Principled Method for Query and Training](https://mlsanity.com/paper/1911.09162/) (2019)
- [Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance](https://mlsanity.com/paper/1805.11897/) (2018)
- [Massively scalable Sinkhorn distances via the Nystr"om method](https://mlsanity.com/paper/1812.05189/) (2018)
- [Rethinking deep active learning: Using unlabeled data at model training](https://mlsanity.com/paper/1911.08177/) (2019)
- [Active One-shot Learning](https://mlsanity.com/paper/1702.06559/) (2017)
- [A Meta-Learning Approach to One-Step Active Learning](https://mlsanity.com/paper/1706.08334/) (2017)
- [Learning under Distributed Weak Supervision](https://mlsanity.com/paper/1606.01100/) (2016)
- [Scenario Reduction Revisited: Fundamental Limits and Guarantees](https://mlsanity.com/paper/1701.04072/) (2017)

## Built on by (most cited first, 30 of 34)

- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Active Learning Through a Covering Lens](https://mlsanity.com/paper/2205.11320/) (2022)
- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Active Finetuning: Exploiting Annotation Budget in the Pretraining-Finetuning Paradigm](https://mlsanity.com/paper/2303.14382/) (2023)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [How Much More Data Do I Need? Estimating Requirements for Downstream Tasks](https://mlsanity.com/paper/2207.01725/) (2022)
- [How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget](https://mlsanity.com/paper/2306.03543/) (2023)
- [Knowledge-Aware Federated Active Learning with Non-IID Data](https://mlsanity.com/paper/2211.13579/) (2022)
- [Generalized Coverage for More Robust Low-Budget Active Learning](https://mlsanity.com/paper/2407.12212/) (2024)
- [A Simple Baseline for Low-Budget Active Learning](https://mlsanity.com/paper/2110.12033/) (2021)
- [Optimizing Data Collection for Machine Learning](https://mlsanity.com/paper/2210.01234/) (2022)
- [Uncertainty Herding: One Active Learning Method for All Label Budgets](https://mlsanity.com/paper/2412.20644/) (2024)
- [Revisiting Active Learning in the Era of Vision Foundation Models](https://mlsanity.com/paper/2401.14555/) (2024)
- [Towards Free Data Selection with General-Purpose Models](https://mlsanity.com/paper/2309.17342/) (2023)
- [Bidirectional Uncertainty-Based Active Learning for Open Set Annotation](https://mlsanity.com/paper/2402.15198/) (2024)
- [Querying Easily Flip-flopped Samples for Deep Active Learning](https://mlsanity.com/paper/2401.09787/) (2024)
- [Active Label Correction for Semantic Segmentation with Foundation Models](https://mlsanity.com/paper/2403.10820/) (2024)
- [You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentation](https://mlsanity.com/paper/2304.11762/) (2023)
- [Active Self-Semi-Supervised Learning for Few Labeled Samples](https://mlsanity.com/paper/2203.04560/) (2022)
- [Feature Alignment: Rethinking Efficient Active Learning via Proxy in the Context of Pre-trained Models](https://mlsanity.com/paper/2403.01101/) (2024)
- [Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarios](https://mlsanity.com/paper/2207.01190/) (2022)
- [NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage](https://mlsanity.com/paper/2306.04099/) (2023)
- [Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play](https://mlsanity.com/paper/2302.05807/) (2023)
- [VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness](https://mlsanity.com/paper/2401.07853/) (2024)
- [Enhancing Semi-Supervised Learning via Representative and Diverse Sample Selection](https://mlsanity.com/paper/2409.11653/) (2024)
- [Active Prompt Learning with Vision-Language Model Priors](https://mlsanity.com/paper/2411.16722/) (2024)
- [VIOLA: Towards Video In-Context Learning with Minimal Annotations](https://mlsanity.com/paper/2601.15549/) (2026)
- [Routing, Cascades, and User Choice for LLMs](https://mlsanity.com/paper/2602.09902/) (2026)
- [Pruning the Unlabeled Data to Improve Semi-Supervised Learning](https://mlsanity.com/paper/2308.14058/) (2023)
- [Scaling-Aware Data Selection for End-to-End Autonomous Driving Systems](https://mlsanity.com/paper/2604.08366/) (2026)

## Abstract (arXiv)

Active learning is the process of training a model with limited labeled data by selecting a core subset of an unlabeled data pool to label. The large scale of data sets used in deep learning forces most sample selection strategies to employ efficient heuristics. This paper introduces an integer optimization problem for selecting a core set that minimizes the discrete Wasserstein distance from the unlabeled pool. We demonstrate that this problem can be tractably solved with a Generalized Benders Decomposition algorithm. Our strategy uses high-quality latent features that can be obtained by unsupervised learning on the unlabeled pool. Numerical results on several data sets show that our optimization approach is competitive with baselines and particularly outperforms them in the low budget regime where less than one percent of the data set is labeled.