# High Dimensional Binary Classification under Label Shift: Phase Transition and Regularization

arXiv: https://arxiv.org/abs/2212.00700 · HTML: https://mlsanity.com/paper/2212.00700/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 80 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [Harmless interpolation of noisy data in regression](https://mlsanity.com/paper/1903.09139/) (2019)
- [A Model of Double Descent for High-dimensional Binary Linear Classification](https://mlsanity.com/paper/1911.05822/) (2019)
- [More Data Can Hurt for Linear Regression: Sample-wise Double Descent](https://mlsanity.com/paper/1912.07242/) (2019)
- [Exact expressions for double descent and implicit regularization via surrogate random design](https://mlsanity.com/paper/1912.04533/) (2019)
- [A Survey of Predictive Modelling under Imbalanced Distributions](https://mlsanity.com/paper/1505.01658/) (2015)
- [Active Learning under Label Shift](https://mlsanity.com/paper/2007.08479/) (2020)
- [A New Look at an Old Problem: A Universal Learning Approach to Linear Regression](https://mlsanity.com/paper/1905.04708/) (2019)
- [A Large Dimensional Study of Regularized Discriminant Analysis Classifiers](https://mlsanity.com/paper/1711.00382/) (2017)

## Abstract (arXiv)

Label Shift has been widely believed to be harmful to the generalization performance of machine learning models. Researchers have proposed many approaches to mitigate the impact of the label shift, e.g., balancing the training data. However, these methods often consider the underparametrized regime, where the sample size is much larger than the data dimension. The research under the overparametrized regime is very limited. To bridge this gap, we propose a new asymptotic analysis of the Fisher Linear Discriminant classifier for binary classification with label shift. Specifically, we prove that there exists a phase transition phenomenon: Under certain overparametrized regime, the classifier trained using imbalanced data outperforms the counterpart with reduced balanced data. Moreover, we investigate the impact of regularization to the label shift: The aforementioned phase transition vanishes as the regularization becomes strong.