mlsanityData to 14 Oct 2026

When Personalization Harms: Reconsidering the Use of Group Attributes in Prediction

arXiv 2206.02058 ↗First posted 4 Jun 2022stat.ML cs.CY cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Machine learning models are often personalized with categorical attributes that are protected, sensitive, self-reported, or costly to acquire. In this work, we show models that are personalized with group attributes can reduce performance at a group level. We propose formal conditions to ensure the "fair use" of group attributes in prediction tasks by training one additional model -- i.e., collective preference guarantees to ensure that each group who provides personal data will receive a tailored gain in performance in return. We present sufficient conditions to ensure fair use in empirical risk minimization and characterize failure modes that lead to fair use violations due to standard practices in model development and deployment. We present a comprehensive empirical study of fair use in clinical prediction tasks. Our results demonstrate the prevalence of fair use violations in practice and illustrate simple interventions to mitigate their harm.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 35 of its 121 references that are in the map. First posted on arXiv 4 Jun 2022.

Builds on

35 of 121 references.

References · 35

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2018Model Cards for Model Reportingcited by 1,080
  3. 2014Certifying and removing disparate impactcited by 715
  4. 2018A Reductions Approach to Fair Classificationcited by 587
  5. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 384
  6. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 369
  7. 2017Multitask learning and benchmarking with clinical time series datacited by 290
  8. 2021Reading Race: AI Recognises Patient's Racial Identity In Medical Imagescited by 156
  9. 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 153
  10. 2018Classification with Fairness Constraints: A Meta-Algorithm with Provable Guaranteescited by 146
  11. 2014From Predictive to Prescriptive Analyticscited by 143
  12. 2020Algorithmic Fairnesscited by 143
23 more
  1. 2020Ethical Machine Learning in Health Carecited by 113
  2. 2019FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learningcited by 93
  3. 2015Supersparse Linear Integer Models for Optimized Medical Scoring Systemscited by 76
  4. 2017From Parity to Preference-based Notions of Fairness in Classificationcited by 60
  5. 2022Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluationcited by 59
  6. 2019Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributionscited by 48
  7. 2021Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffscited by 48
  8. 2016Learning Optimized Risk Scorescited by 47
  9. 2019Fair Classification and Social Welfarecited by 40
  10. 2022The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regressioncited by 28
  11. 2020Model Distillation for Revenue Optimization: Interpretable Personalized Pricingcited by 24
  12. 2018Creating Fair Models of Atherosclerotic Cardiovascular Disease Riskcited by 23
  13. 2021Online certification of preference-based fairness for personalized recommender systemscited by 22
  14. 2022An Algorithmic Framework for Bias Bountiescited by 21
  15. 2022Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendationscited by 20
  16. 2019Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcarecited by 16
  17. 2019Preference-Informed Fairnesscited by 15
  18. 2020To Split or Not to Split: The Impact of Disparate Treatment in Classificationcited by 14
  19. 2020Fair Policy Targetingcited by 12
  20. 2018Envy-Free Classificationcited by 11
  21. 2021Learning Optimal Predictive Checklistscited by 10
  22. 2021Disaggregated Interventions to Reduce Inequalitycited by 5
  23. 2018Unleashing Linear Optimizers for Group-Fair Learning and Optimizationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2024Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposiumcited by 3
  2. 2025Regretful Decisions under Label Noisecited by 1
  3. 2025"Who experiences large model decay and why?" A Hierarchical Framework for Diagnosing Heterogeneous Performance Driftcited by 1
  4. 2024Considerations for Distribution Shift Robustness of Diagnostic Models in Healthcarecited by 0
  5. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  6. 2026Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Educationcited by 0