# Benchmarking Multi-Domain Active Learning on Image Classification

arXiv: https://arxiv.org/abs/2312.00364 · HTML: https://mlsanity.com/paper/2312.00364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Leveraging Unlabeled Data to Predict Out-of-Distribution Performance](https://mlsanity.com/paper/2201.04234/) (2022)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [Active Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07848/) (2019)
- [Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings](https://mlsanity.com/paper/2010.08666/) (2020)
- [Adaptive Methods for Real-World Domain Generalization](https://mlsanity.com/paper/2103.15796/) (2021)
- [Multinomial Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/1802.05694/) (2018)
- [Active Sampling for Min-Max Fairness](https://mlsanity.com/paper/2006.06879/) (2020)
- [Active Learning Helps Pretrained Models Learn the Intended Task](https://mlsanity.com/paper/2204.08491/) (2022)
- [Fair Active Learning](https://mlsanity.com/paper/2001.01796/) (2020)
- [Adaptive Sampling for Minimax Fair Classification](https://mlsanity.com/paper/2103.00755/) (2021)
- [Conditional Adversarial Networks for Multi-Domain Text Classification](https://mlsanity.com/paper/2102.10176/) (2021)
- [On the Relationship between Data Efficiency and Error for Uncertainty Sampling](https://mlsanity.com/paper/1806.06123/) (2018)
- [On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks](https://mlsanity.com/paper/2010.05103/) (2020)
- [Is margin all you need? An extensive empirical study of active learning on tabular data](https://mlsanity.com/paper/2210.03822/) (2022)
- [Multi-Domain Active Learning: Literature Review and Comparative Study](https://mlsanity.com/paper/2106.13516/) (2021)

## Abstract (arXiv)

Active learning aims to enhance model performance by strategically labeling informative data points. While extensively studied, its effectiveness on large-scale, real-world datasets remains underexplored. Existing research primarily focuses on single-source data, ignoring the multi-domain nature of real-world data. We introduce a multi-domain active learning benchmark to bridge this gap. Our benchmark demonstrates that traditional single-domain active learning strategies are often less effective than random selection in multi-domain scenarios. We also introduce CLIP-GeoYFCC, a novel large-scale image dataset built around geographical domains, in contrast to existing genre-based domain datasets. Analysis on our benchmark shows that all multi-domain strategies exhibit significant tradeoffs, with no strategy outperforming across all datasets or all metrics, emphasizing the need for future research.