# Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture

arXiv: https://arxiv.org/abs/2411.02047 · HTML: https://mlsanity.com/paper/2411.02047/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 72 references that are in the map.
- First posted on arXiv 4 Nov 2024.

## Builds on (most cited first, 30 of 39)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Regularized Learning for Domain Adaptation under Label Shifts](https://mlsanity.com/paper/1903.09734/) (2019)
- [A Unified View of Label Shift Estimation](https://mlsanity.com/paper/2003.07554/) (2020)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation](https://mlsanity.com/paper/2207.14191/) (2022)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Open-environment Machine Learning](https://mlsanity.com/paper/2206.00423/) (2022)
- [Online Adaptation to Label Distribution Shift](https://mlsanity.com/paper/2107.04520/) (2021)
- [Mind the Label Shift of Augmentation-based Graph OOD Generalization](https://mlsanity.com/paper/2303.14859/) (2023)
- [RLSbench: Domain Adaptation Under Relaxed Label Shift](https://mlsanity.com/paper/2302.03020/) (2023)
- [Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error](https://mlsanity.com/paper/2106.09613/) (2021)
- [Conformal Prediction for Federated Uncertainty Quantification Under Label Shift](https://mlsanity.com/paper/2306.05131/) (2023)
- [Adapting to Online Label Shift with Provable Guarantees](https://mlsanity.com/paper/2207.02121/) (2022)
- [Aggregating From Multiple Target-Shifted Sources](https://mlsanity.com/paper/2105.04051/) (2021)
- [Active Learning under Label Shift](https://mlsanity.com/paper/2007.08479/) (2020)
- [Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts](https://mlsanity.com/paper/2308.08810/) (2023)
- [Fisher consistency for prior probability shift](https://mlsanity.com/paper/1701.05512/) (2017)
- [Minimax optimal approaches to the label shift problem in non-parametric settings](https://mlsanity.com/paper/2003.10443/) (2020)
- [Optimal Transport for Conditional Domain Matching and Label Shift](https://mlsanity.com/paper/2006.08161/) (2020)
- [Mapping conditional distributions for domain adaptation under generalized target shift](https://mlsanity.com/paper/2110.15057/) (2021)
- [An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift](https://mlsanity.com/paper/2202.12123/) (2022)
- [Imbalanced Semi-supervised Learning with Bias Adaptive Classifier](https://mlsanity.com/paper/2207.13856/) (2022)
- [Quantification under prior probability shift: the ratio estimator and its extensions](https://mlsanity.com/paper/1807.03929/) (2018)

## Abstract (arXiv)

As a prominent challenge in addressing real-world issues within a dynamic environment, label shift, which refers to the learning setting where the source (training) and target (testing) label distributions do not match, has recently received increasing attention. Existing label shift methods solely use unlabeled target samples to estimate the target label distribution, and do not involve them during the classifier training, resulting in suboptimal utilization of available information. One common solution is to directly blend the source and target distributions during the training of the target classifier. However, we illustrate the theoretical deviation and limitations of the direct distribution mixture in the label shift setting. To tackle this crucial yet unexplored issue, we introduce the concept of aligned distribution mixture, showcasing its theoretical optimality and generalization error bounds. By incorporating insights from generalization theory, we propose an innovative label shift framework named as Aligned Distribution Mixture (ADM). Within this framework, we enhance four typical label shift methods by introducing modifications to the classifier training process. Furthermore, we also propose a one-step approach that incorporates a pioneering coupling weight estimation strategy. Considering the distinctiveness of the proposed one-step approach, we develop an efficient bi-level optimization strategy. Experimental results demonstrate the effectiveness of our approaches, together with their effectiveness in COVID-19 diagnosis applications.