# Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory

arXiv: https://arxiv.org/abs/2205.05512 · HTML: https://mlsanity.com/paper/2205.05512/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 6 of 6)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity](https://mlsanity.com/paper/1809.03400/) (2018)
- [Fairness in machine learning: against false positive rate equality as a measure of fairness](https://mlsanity.com/paper/2007.02890/) (2020)

## Built on by (most cited first, 6 of 6)

- [What Is Fairness? On the Role of Protected Attributes and Fictitious Worlds](https://mlsanity.com/paper/2205.09622/) (2022)
- [Fairness in Ranking under Disparate Uncertainty](https://mlsanity.com/paper/2309.01610/) (2023)
- [An Operational Perspective to Fairness Interventions: Where and How to Intervene](https://mlsanity.com/paper/2302.01574/) (2023)
- [Geospatial Disparities: A Case Study on Real Estate Prices in Paris](https://mlsanity.com/paper/2401.16197/) (2024)
- [Does calibration mean what they say it means; or, the reference class problem rises again](https://mlsanity.com/paper/2412.16769/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)

## Abstract (arXiv)

In this paper, we provide a moral analysis of two criteria of statistical fairness debated in the machine learning literature: 1) calibration between groups and 2) equality of false positive and false negative rates between groups. In our paper, we focus on moral arguments in support of either measure. The conflict between group calibration vs. false positive and false negative rate equality is one of the core issues in the debate about group fairness definitions among practitioners. For any thorough moral analysis, the meaning of the term fairness has to be made explicit and defined properly. For our paper, we equate fairness with (non-)discrimination, which is a legitimate understanding in the discussion about group fairness. More specifically, we equate it with prima facie wrongful discrimination in the sense this is used in Prof. Lippert-Rasmussen's treatment of this definition. In this paper, we argue that a violation of group calibration may be unfair in some cases, but not unfair in others. This is in line with claims already advanced in the literature, that algorithmic fairness should be defined in a way that is sensitive to context. The most important practical implication is that arguments based on examples in which fairness requires between-group calibration, or equality in the false-positive/false-negative rates, do no generalize. For it may be that group calibration is a fairness requirement in one case, but not in another.