# Supervised Algorithmic Fairness in Distribution Shifts: A Survey

arXiv: https://arxiv.org/abs/2402.01327 · HTML: https://mlsanity.com/paper/2402.01327/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018)
- [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019)
- [Robust Fairness under Covariate Shift](https://mlsanity.com/paper/2010.05166/) (2020)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [Causal Modeling for Fairness in Dynamical Systems](https://mlsanity.com/paper/1909.09141/) (2019)
- [Diagnosing failures of fairness transfer across distribution shift in real-world medical settings](https://mlsanity.com/paper/2202.01034/) (2022)
- [A Distributionally Robust Approach to Fair Classification](https://mlsanity.com/paper/2007.09530/) (2020)
- [Transfer of Machine Learning Fairness across Domains](https://mlsanity.com/paper/1906.09688/) (2019)
- [Ensuring Fairness Beyond the Training Data](https://mlsanity.com/paper/2007.06029/) (2020)
- [Learning Fair Representation via Distributional Contrastive Disentanglement](https://mlsanity.com/paper/2206.08743/) (2022)
- [Fairness Transferability Subject to Bounded Distribution Shift](https://mlsanity.com/paper/2206.00129/) (2022)
- [FairOD: Fairness-aware Outlier Detection](https://mlsanity.com/paper/2012.03063/) (2020)
- [Fairness and Accuracy under Domain Generalization](https://mlsanity.com/paper/2301.13323/) (2023)
- [Transferring Fairness under Distribution Shifts via Fair Consistency Regularization](https://mlsanity.com/paper/2206.12796/) (2022)
- [Deep Clustering based Fair Outlier Detection](https://mlsanity.com/paper/2106.05127/) (2021)
- [Fairness-Aware Online Meta-learning](https://mlsanity.com/paper/2108.09435/) (2021)
- [Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference](https://mlsanity.com/paper/2107.10931/) (2021)
- [Adaptive Fairness-Aware Online Meta-Learning for Changing Environments](https://mlsanity.com/paper/2205.11264/) (2022)
- [Ensuring Fairness under Prior Probability Shifts](https://mlsanity.com/paper/2005.03474/) (2020)
- [Fairness-enhancing interventions in stream classification](https://mlsanity.com/paper/1907.07223/) (2019)
- [Improving Fair Training under Correlation Shifts](https://mlsanity.com/paper/2302.02323/) (2023)
- [Towards Fair Disentangled Online Learning for Changing Environments](https://mlsanity.com/paper/2306.01007/) (2023)

## Built on by (most cited first, 24 of 24)

- [Multi-Modal Foundation Models for Computational Pathology: A Survey](https://mlsanity.com/paper/2503.09091/) (2025)
- [Learning Fair Invariant Representations under Covariate and Correlation Shifts Simultaneously](https://mlsanity.com/paper/2408.09312/) (2024)
- [The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment](https://mlsanity.com/paper/2509.17100/) (2025)
- [GDDA: Semantic OOD Detection on Graphs under Covariate Shift via Score-Based Diffusion Models](https://mlsanity.com/paper/2410.17526/) (2024)
- [Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts](https://mlsanity.com/paper/2602.04398/) (2026)
- [FADE: Towards Fairness-aware Generation for Domain Generalization via Classifier-Guided Score-based Diffusion Models](https://mlsanity.com/paper/2406.09495/) (2024)
- [MLDGG: Meta-Learning for Domain Generalization on Graphs](https://mlsanity.com/paper/2411.12913/) (2024)
- [FEED: Fairness-Enhanced Meta-Learning for Domain Generalization](https://mlsanity.com/paper/2411.01316/) (2024)
- [Object Hallucination-Free Reinforcement Unlearning for Vision-Language Models](https://mlsanity.com/paper/2605.08031/) (2026)
- [Hypergraph-Based Dynamic Graph Node Classification](https://mlsanity.com/paper/2412.20321/) (2024)
- [Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies](https://mlsanity.com/paper/2505.22829/) (2025)
- [BioPro: Towards Difference-Aware Gender Fairness for Vision-Language Models](https://mlsanity.com/paper/2512.00807/) (2025)
- [Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalization](https://mlsanity.com/paper/2411.07392/) (2024)
- [Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains](https://mlsanity.com/paper/2509.00658/) (2025)
- [LLM-Enhanced Energy Contrastive Learning for Out-of-Distribution Detection in Text-Attributed Graphs](https://mlsanity.com/paper/2603.20293/) (2026)
- [On the Robustness of Machine Unlearning for Vision-Language Models](https://mlsanity.com/paper/2605.26992/) (2026)
- [Fairness-Aware Low-Rank Representation Fine-Tuning](https://mlsanity.com/paper/2503.05684/) (2025)
- [Out-of-Distribution Detection in Heterogeneous Graphs via Energy Propagation](https://mlsanity.com/paper/2505.03774/) (2025)
- [Failing on Bias Mitigation: A Case Study on the Challenges of Fairness in Government Data](https://mlsanity.com/paper/2601.17054/) (2026)
- [Causal Pre-training Under the Fairness Lens: An Empirical Study of TabPFN](https://mlsanity.com/paper/2601.17912/) (2026)
- [MARLIN: Multi-Agent Reinforcement Learning for Incremental DAG Discovery](https://mlsanity.com/paper/2603.20295/) (2026)
- [ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models](https://mlsanity.com/paper/2605.18879/) (2026)
- [Evaluating Model Retraining under Drift: Paired Comparisons of Cumulative Subgroup Disparity](https://mlsanity.com/paper/2609.09788/) (2026)
- [FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models](https://mlsanity.com/paper/2610.05790/) (2026)

## Abstract (arXiv)

Supervised fairness-aware machine learning under distribution shifts is an emerging field that addresses the challenge of maintaining equitable and unbiased predictions when faced with changes in data distributions from source to target domains. In real-world applications, machine learning models are often trained on a specific dataset but deployed in environments where the data distribution may shift over time due to various factors. This shift can lead to unfair predictions, disproportionately affecting certain groups characterized by sensitive attributes, such as race and gender. In this survey, we provide a summary of various types of distribution shifts and comprehensively investigate existing methods based on these shifts, highlighting six commonly used approaches in the literature. Additionally, this survey lists publicly available datasets and evaluation metrics for empirical studies. We further explore the interconnection with related research fields, discuss the significant challenges, and identify potential directions for future studies.