mlsanityData to 14 Oct 2026

Diagnosing failures of fairness transfer across distribution shift in real-world medical settings

arXiv 2202.01034 ↗First posted 2 Feb 2022cs.LG cs.CY stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the structure of the shift. Despite this, there has been little discussion of how to empirically assess the structure of a distribution shift that one is encountering in practice. In this work, we adopt a causal framing to motivate conditional independence tests as a key tool for characterizing distribution shifts. Using our approach in two medical applications, we show that this knowledge can help diagnose failures of fairness transfer, including cases where real-world shifts are more complex than is often assumed in the literature. Based on these results, we discuss potential remedies at each step of the machine learning pipeline.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 49 of its 101 references that are in the map. First posted on arXiv 2 Feb 2022.

Builds on

49 of 101 references.

References · 49

  1. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  2. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  3. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  4. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  5. 2018A Reductions Approach to Fair Classificationcited by 587
  6. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  7. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  8. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 379
  9. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 369
  10. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 366
  11. 2020Environment Inference for Invariant Learningcited by 306
  12. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
37 more
  1. 2020A Brief Review of Domain Adaptationcited by 288
  2. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  3. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  4. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  5. 2020Fairness in Machine Learningcited by 194
  6. 2019Causality matters in medical imagingcited by 193
  7. 2018Path-Specific Counterfactual Fairnesscited by 181
  8. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  9. 2017Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributionscited by 126
  10. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  11. 2020A Unified View of Label Shift Estimationcited by 119
  12. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  13. 2020Ethical Machine Learning in Health Carecited by 113
  14. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 104
  15. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  16. 2019Training individually fair ML models with Sensitive Subspace Robustnesscited by 73
  17. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  18. 2020Fairness Through Robustness: Investigating Robustness Disparity in Deep Learningcited by 56
  19. 2018Residual Unfairness in Fair Machine Learning from Prejudiced Datacited by 49
  20. 2019Causal Modeling for Fairness in Dynamical Systemscited by 47
  21. 2020An Empirical Characterization of Fair Machine Learning For Clinical Risk Predictioncited by 45
  22. 2020A Distributionally Robust Approach to Fair Classificationcited by 43
  23. 2019Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Taskscited by 39
  24. 2019Transfer of Machine Learning Fairness across Domainscited by 36
  25. 2020Fairness and Robustness in Invariant Learning: A Case Study in Toxicity Classificationcited by 32
  26. 2021An Empirical Framework for Domain Generalization in Clinical Settingscited by 32
  27. 2019Fairness Warnings and Fair-MAML: Learning Fairly with Minimal Datacited by 23
  28. 2021Technical Challenges for Training Fair Neural Networkscited by 20
  29. 2021A Near-Optimal Algorithm for Debiasing Trained Machine Learning Modelscited by 19
  30. 2020Individual Fairness Revisited: Transferring Techniques from Adversarial Robustnesscited by 17
  31. 2019Permutation Weightingcited by 14
  32. 2020Fair Meta-Learning For Few-Shot Classificationcited by 13
  33. 2019GradMask: Reduce Overfitting by Regularizing Saliencycited by 12
  34. 2020Partial Identifiability in Discrete Data With Measurement Errorcited by 10
  35. 2019Learning Fair and Transferable Representationscited by 6
  36. 2017Discriminatory Transfercited by 4
  37. 2019Population-aware Hierarchical Bayesian Domain Adaptation via Multiple-component Invariant Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 16; 2025: 10; 2026: 4.

45 papers.

Sort:

By area:

  1. 2023Generative models improve fairness of medical classifiers under distribution shiftscited by 90
  2. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 85
  3. 2022Detecting Shortcut Learning for Fair Medical AI using Shortcut Testingcited by 52
  4. 2024Supervised Algorithmic Fairness in Distribution Shifts: A Surveycited by 24
  5. 2024The Data Addition Dilemmacited by 15
  6. 2023No Fair Lunch: A Causal Perspective on Dataset Bias in Machine Learning for Medical Imagingcited by 11
  7. 2023Designing Long-term Group Fair Policies in Dynamical Systemscited by 9
  8. 2024Mind the Graph When Balancing Data for Fairness or Robustnesscited by 9
  9. 2024Rethinking Fair Representation Learning for Performance-Sensitive Taskscited by 8
  10. 2022A unified framework for dataset shift diagnosticscited by 7
  11. 2023Algorithmic Fairness Generalization under Covariate and Dependence Shifts Simultaneouslycited by 7
  12. 2023Explanation Shift: How Did the Distribution Shift Impact the Model?cited by 6
33 more
  1. 2024Proxy Methods for Domain Adaptationcited by 6
  2. 2025Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairnesscited by 6
  3. 2025Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradationcited by 6
  4. 2023Classical Out-of-Distribution Detection Methods Benchmark in Text Classification Taskscited by 5
  5. 2024The Case for Globalizing Fairness: A Mixed Methods Study on Colonialism, AI, and Health in Africacited by 5
  6. 2024Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounderscited by 5
  7. 2025When Shift Happens - Confounding Is to Blamecited by 5
  8. 2023The Limits of Fair Medical Imaging AI In The Wildcited by 4
  9. 2024Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequitiescited by 3
  10. 2025Fair Foundation Models for Medical Image Analysis: Challenges and Perspectivescited by 3
  11. 2023Adapting Machine Learning Diagnostic Models to New Populations Using a Small Amount of Data: Results from Clinical Neurosciencecited by 2
  12. 2024Regulating AI Adaptation: An Analysis of AI Medical Device Updatescited by 2
  13. 2024Open Challenges on Fairness of Artificial Intelligence in Medical Imaging Applicationscited by 2
  14. 2023Unsupervised bias discovery in medical image segmentationcited by 1
  15. 2023Mixture Data for Training Cannot Ensure Out-of-distribution Generalizationcited by 1
  16. 2024A Note on Bias to Completecited by 1
  17. 2024Using Backbone Foundation Model for Evaluating Fairness in Chest Radiography Without Demographic Datacited by 1
  18. 2025Understanding-informed Bias Mitigation for Fair CMR Segmentationcited by 1
  19. 2025Localising Shortcut Learning in Pixel Space via Ordinal Scoring Correlations for Attribution Representations (OSCAR)cited by 1
  20. 2026Transfer Learning Through Conditional Quantile Matchingcited by 1
  21. 2026Understanding Sources of Demographic Predictability in Brain MRI via Disentangling Anatomy and Contrastcited by 1
  22. 2023Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africacited by 0
  23. 2023Transferring Fairness using Multi-Task Learning with Limited Demographic Informationcited by 0
  24. 2023An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasetscited by 0
  25. 2024Accurately Classifying Out-Of-Distribution Data in Facial Recognitioncited by 0
  26. 2024Self-Supervised Learning for Building Robust Pediatric Chest X-ray Classification Modelscited by 0
  27. 2024Revisiting Technical Bias Mitigation Strategiescited by 0
  28. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  29. 2025You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Timecited by 0
  30. 2025A global log for medical AIcited by 0
  31. 2025Representation Invariance and Allocation: When Subgroup Balance Matterscited by 0
  32. 2026Dense Feature Learning via Linear Structure Preservation in Medical Datacited by 0
  33. 2026Large language models in medical time series analysiscited by 0