# When fairness metrics fail: A utility-based perspective on varepsilon-fairness

arXiv: https://arxiv.org/abs/2405.09360 · HTML: https://mlsanity.com/paper/2405.09360/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 15 May 2024.

## Builds on (most cited first, 14 of 14)

- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Delayed Impact of Fair Machine Learning](https://mlsanity.com/paper/1803.04383/) (2018)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [Algorithms for Fairness in Sequential Decision Making](https://mlsanity.com/paper/1901.08568/) (2019)
- [A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity](https://mlsanity.com/paper/1809.03400/) (2018)
- [Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife](https://mlsanity.com/paper/1311.4555/) (2013)
- [Confidence Intervals for Testing Disparate Impact in Fair Learning](https://mlsanity.com/paper/1807.06362/) (2018)
- [Protecting the Protected Group: Circumventing Harmful Fairness](https://mlsanity.com/paper/1905.10546/) (2019)
- [Stochastic Optimization for Spectral Risk Measures](https://mlsanity.com/paper/2212.05149/) (2022)
- [A Discussion of Discrimination and Fairness in Insurance Pricing](https://mlsanity.com/paper/2209.00858/) (2022)
- [On Modeling Human Perceptions of Allocation Policies with Uncertain Outcomes](https://mlsanity.com/paper/2103.05827/) (2021)
- [Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing](https://mlsanity.com/paper/2312.03867/) (2023)

## Abstract (arXiv)

Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics need not reflect the consequences of decisions for the affected individuals and groups. We develop a utility-based framework that incorporates these consequences into the assessment of fairness. Our main result shows that a decision-making process can satisfy varepsilon-fairness while nevertheless being maximally unfair once the utilities associated with its outcomes are taken into account. To address applications in which information on false negatives is unavailable, we also formulate a reduced setting that retains the essential elements of the utility-based fairness assessment. We illustrate the framework through two applications: college admissions and credit-risk assessment. In both cases, probabilistic metrics may classify a decision-making process as approximately fair even though the corresponding utility outcomes are highly unequal. In the college-admissions example, our analysis shows that improving completion rates is necessary to achieve equality of utility across groups, while in the mortgage example, mitigating unfairness requires not only adjusting approval rates but also reducing the adverse consequences of default. These findings demonstrate that fairness assessments should account not only for the probabilities of different decisions but also for the consequences of those decisions.