# Counterpart Fairness -- Addressing Systematic between-group Differences in Fairness Evaluation

arXiv: https://arxiv.org/abs/2305.18160 · HTML: https://mlsanity.com/paper/2305.18160/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 86 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 28 of 28)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Inherent Tradeoffs in Learning Fair Representations](https://mlsanity.com/paper/1906.08386/) (2019)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices](https://mlsanity.com/paper/1807.00787/) (2018)
- [Conditional Learning of Fair Representations](https://mlsanity.com/paper/1910.07162/) (2019)
- [Learning Certified Individually Fair Representations](https://mlsanity.com/paper/2002.10312/) (2020)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [Two Simple Ways to Learn Individual Fairness Metrics from Data](https://mlsanity.com/paper/2006.11439/) (2020)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [On the folded normal distribution](https://mlsanity.com/paper/1402.3559/) (2014)
- [Creating Fair Models of Atherosclerotic Cardiovascular Disease Risk](https://mlsanity.com/paper/1809.04663/) (2018)
- [An Empirical Study on Learning Fairness Metrics for COMPAS Data with Human Supervision](https://mlsanity.com/paper/1910.10255/) (2019)
- [Statistical inference for individual fairness](https://mlsanity.com/paper/2103.16714/) (2021)
- [IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms](https://mlsanity.com/paper/1810.02658/) (2018)

## Abstract (arXiv)

When using machine learning to aid decision-making, it is critical to ensure that an algorithmic decision is fair and does not discriminate against specific individuals/groups, particularly those from underprivileged populations. Existing group fairness methods aim to ensure equal outcomes (such as loan approval rates) across groups delineated by protected variables like race or gender. However, in cases where systematic differences between groups play a significant role in outcomes, these methods may overlook the influence of non-protected variables that can systematically vary across groups. These confounding factors can affect fairness evaluations, making it challenging to assess whether disparities are due to discrimination or inherent differences. Therefore, we recommend a more refined and comprehensive fairness index that accounts for both the systematic differences within groups and the multifaceted, intertwined confounding effects. The proposed index evaluates fairness on counterparts (pairs of individuals who are similar with respect to the task of interest but from different groups), whose group identities cannot be distinguished algorithmically by exploring confounding factors. To identify counterparts, we developed a two-step matching method inspired by propensity score and metric learning. In addition, we introduced a counterpart-based statistical fairness index, called Counterpart Fairness (CFair), to assess the fairness of machine learning models. Empirical results on the MIMIC and COMPAS datasets indicate that standard group-based fairness metrics may not adequately inform about the degree of unfairness present in predictions, as revealed through CFair.