# Improving Fair Training under Correlation Shifts

arXiv: https://arxiv.org/abs/2302.02323 · HTML: https://mlsanity.com/paper/2302.02323/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 66 references that are in the map.
- First posted on arXiv 5 Feb 2023.

## Builds on (most cited first, 29 of 29)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [FairBatch: Batch Selection for Model Fairness](https://mlsanity.com/paper/2012.01696/) (2020)
- [Conditional Learning of Fair Representations](https://mlsanity.com/paper/1910.07162/) (2019)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification](https://mlsanity.com/paper/1906.05082/) (2019)
- [Poisoning Attacks on Algorithmic Fairness](https://mlsanity.com/paper/2004.07401/) (2020)
- [Robust Fairness under Covariate Shift](https://mlsanity.com/paper/2010.05166/) (2020)
- [Two-Player Games for Efficient Non-Convex Constrained Optimization](https://mlsanity.com/paper/1804.06500/) (2018)
- [FR-Train: A Mutual Information-Based Approach to Fair and Robust Training](https://mlsanity.com/paper/2002.10234/) (2020)
- [Exacerbating Algorithmic Bias through Fairness Attacks](https://mlsanity.com/paper/2012.08723/) (2020)
- [Fairness Transferability Subject to Bounded Distribution Shift](https://mlsanity.com/paper/2206.00129/) (2022)
- [Does enforcing fairness mitigate biases caused by subpopulation shift?](https://mlsanity.com/paper/2011.03173/) (2020)
- [Transferring Fairness under Distribution Shifts via Fair Consistency Regularization](https://mlsanity.com/paper/2206.12796/) (2022)
- [General Heuristics for Nonconvex Quadratically Constrained Quadratic Programming](https://mlsanity.com/paper/1703.07870/) (2017)
- [Fair When Trained, Unfair When Deployed: Observable Fairness Measures are Unstable in Performative Prediction Settings](https://mlsanity.com/paper/2202.05049/) (2022)

## Built on by (most cited first, 14 of 14)

- [Supervised Algorithmic Fairness in Distribution Shifts: A Survey](https://mlsanity.com/paper/2402.01327/) (2024)
- [Fairness Without Harm: An Influence-Guided Active Sampling Approach](https://mlsanity.com/paper/2402.12789/) (2024)
- [Mind the Graph When Balancing Data for Fairness or Robustness](https://mlsanity.com/paper/2406.17433/) (2024)
- [Algorithmic Fairness Generalization under Covariate and Dependence Shifts Simultaneously](https://mlsanity.com/paper/2311.13816/) (2023)
- [Learning Fair Invariant Representations under Covariate and Correlation Shifts Simultaneously](https://mlsanity.com/paper/2408.09312/) (2024)
- [LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views](https://mlsanity.com/paper/2402.04644/) (2024)
- [On the Maximal Local Disparity of Fairness-Aware Classifiers](https://mlsanity.com/paper/2406.03255/) (2024)
- [Stress-Testing ML Pipelines with Adversarial Data Corruption](https://mlsanity.com/paper/2506.01230/) (2025)
- [Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains](https://mlsanity.com/paper/2509.00658/) (2025)
- [dsld: A Socially Relevant Tool for Teaching Statistics](https://mlsanity.com/paper/2411.04228/) (2024)
- [TowerDebias: A Novel Unfairness Removal Method Based on the Tower Property](https://mlsanity.com/paper/2411.08297/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Time](https://mlsanity.com/paper/2503.07066/) (2025)
- [Whence Is A Model Fair? Fixing Fairness Bugs via Propensity Score Matching](https://mlsanity.com/paper/2504.17066/) (2025)

## Abstract (arXiv)

Model fairness is an essential element for Trustworthy AI. While many techniques for model fairness have been proposed, most of them assume that the training and deployment data distributions are identical, which is often not true in practice. In particular, when the bias between labels and sensitive groups changes, the fairness of the trained model is directly influenced and can worsen. We make two contributions for solving this problem. First, we analytically show that existing in-processing fair algorithms have fundamental limits in accuracy and group fairness. We introduce the notion of correlation shifts, which can explicitly capture the change of the above bias. Second, we propose a novel pre-processing step that samples the input data to reduce correlation shifts and thus enables the in-processing approaches to overcome their limitations. We formulate an optimization problem for adjusting the data ratio among labels and sensitive groups to reflect the shifted correlation. A key benefit of our approach lies in decoupling the roles of pre- and in-processing approaches: correlation adjustment via pre-processing and unfairness mitigation on the processed data via in-processing. Experiments show that our framework effectively improves existing in-processing fair algorithms w.r.t. accuracy and fairness, both on synthetic and real datasets.