# Active Learning under Label Shift

arXiv: https://arxiv.org/abs/2007.08479 · HTML: https://mlsanity.com/paper/2007.08479/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2020.

## Built on by (most cited first, 23 of 23)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Estimating and Explaining Model Performance When Both Covariates and Labels Shift](https://mlsanity.com/paper/2209.08436/) (2022)
- [Optimizing Data Collection for Machine Learning](https://mlsanity.com/paper/2210.01234/) (2022)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Did the Model Change? Efficiently Assessing Machine Learning API Shifts](https://mlsanity.com/paper/2107.14203/) (2021)
- [Active Learning Over Multiple Domains in Natural Language Tasks](https://mlsanity.com/paper/2202.00254/) (2022)
- [Active Learning with Expected Error Reduction](https://mlsanity.com/paper/2211.09283/) (2022)
- [Bayesian Active Learning for Sim-to-Real Robotic Perception](https://mlsanity.com/paper/2109.11547/) (2021)
- [Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions](https://mlsanity.com/paper/2211.11798/) (2022)
- [On the Pros and Cons of Active Learning for Moral Preference Elicitation](https://mlsanity.com/paper/2407.18889/) (2024)
- [Corruption Robust Active Learning](https://mlsanity.com/paper/2106.11220/) (2021)
- [Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarios](https://mlsanity.com/paper/2207.01190/) (2022)
- [ASPEST: Bridging the Gap Between Active Learning and Selective Prediction](https://mlsanity.com/paper/2304.03870/) (2023)
- [Distributionally Robust Active Learning for Gaussian Process Regression](https://mlsanity.com/paper/2502.16870/) (2025)
- [Generator Assisted Mixture of Experts For Feature Acquisition in Batch](https://mlsanity.com/paper/2312.12574/) (2023)
- [LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Training](https://mlsanity.com/paper/2411.15217/) (2024)
- [Open Set Label Shift with Test Time Out-of-Distribution Reference](https://mlsanity.com/paper/2505.05868/) (2025)
- [Bayesian-based Online Label Shift Estimation with Dynamic Dirichlet Priors](https://mlsanity.com/paper/2511.18615/) (2025)
- [Discovery of Feasible 3D Printing Configurations for Metal Alloys via AI-driven Adaptive Experimental Design](https://mlsanity.com/paper/2601.17587/) (2026)
- [High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization](https://mlsanity.com/paper/2212.00700/) (2022)
- [Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture](https://mlsanity.com/paper/2411.02047/) (2024)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [When Imbalance Comes Twice: Active Learning under Simulated Class Imbalance and Label Shift in Binary Semantic Segmentation](https://mlsanity.com/paper/2601.06209/) (2026)

## Abstract (arXiv)

We address the problem of active learning under label shift: when the class proportions of source and target domains differ. We introduce a "medial distribution" to incorporate a tradeoff between importance weighting and class-balanced sampling and propose their combined usage in active learning. Our method is known as Mediated Active Learning under Label Shift (MALLS). It balances the bias from class-balanced sampling and the variance from importance weighting. We prove sample complexity and generalization guarantees for MALLS which show active learning reduces asymptotic sample complexity even under arbitrary label shift. We empirically demonstrate MALLS scales to high-dimensional datasets and can reduce the sample complexity of active learning by 60% in deep active learning tasks.