# Retiring DeltaDP: New Distribution-Level Metrics for Demographic Parity

arXiv: https://arxiv.org/abs/2301.13443 · HTML: https://mlsanity.com/paper/2301.13443/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 72 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 30 of 38)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Flexibly Fair Representation Learning by Disentanglement](https://mlsanity.com/paper/1906.02589/) (2019)
- [Censoring Representations with an Adversary](https://mlsanity.com/paper/1511.05897/) (2015)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information](https://mlsanity.com/paper/2009.01454/) (2020)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Towards a Unified Framework for Fair and Stable Graph Representation Learning](https://mlsanity.com/paper/2102.13186/) (2021)
- [Fair Mixup: Fairness via Interpolation](https://mlsanity.com/paper/2103.06503/) (2021)
- [Fair Regression with Wasserstein Barycenters](https://mlsanity.com/paper/2006.07286/) (2020)
- [Training individually fair ML models with Sensitive Subspace Robustness](https://mlsanity.com/paper/1907.00020/) (2019)
- [Fairness in Credit Scoring: Assessment, Implementation and Profit Implications](https://mlsanity.com/paper/2103.01907/) (2021)
- [Learning to Pivot with Adversarial Networks](https://mlsanity.com/paper/1611.01046/) (2016)
- [Two Simple Ways to Learn Individual Fairness Metrics from Data](https://mlsanity.com/paper/2006.11439/) (2020)
- [How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairness](https://mlsanity.com/paper/1811.03654/) (2018)
- [Fairness Metrics: A Comparative Analysis](https://mlsanity.com/paper/2001.07864/) (2020)
- [The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric](https://mlsanity.com/paper/1902.05826/) (2019)
- [SenSeI: Sensitive Set Invariance for Enforcing Individual Fairness](https://mlsanity.com/paper/2006.14168/) (2020)
- [FMP: Toward Fair Graph Message Passing against Topology Bias](https://mlsanity.com/paper/2202.04187/) (2022)
- [Fairness Reprogramming](https://mlsanity.com/paper/2209.10222/) (2022)
- [Transferring Fairness under Distribution Shifts via Fair Consistency Regularization](https://mlsanity.com/paper/2206.12796/) (2022)
- [Counterfactual Fairness with Partially Known Causal Graph](https://mlsanity.com/paper/2205.13972/) (2022)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)

## Built on by (most cited first, 4 of 4)

- [FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods](https://mlsanity.com/paper/2306.09468/) (2023)
- [Uplift modeling with continuous treatments: A predict-then-optimize approach](https://mlsanity.com/paper/2412.09232/) (2024)
- [Decision-centric fairness: Evaluation and optimization for resource allocation problems](https://mlsanity.com/paper/2504.20642/) (2025)
- [Balancing Fairness, Privacy, and Accuracy: A Multitask Adversarial Framework for Centralized Data-Driven Systems](https://mlsanity.com/paper/2605.24458/) (2026)

## Abstract (arXiv)

Demographic parity is the most widely recognized measure of group fairness in machine learning, which ensures equal treatment of different demographic groups. Numerous works aim to achieve demographic parity by pursuing the commonly used metric Delta DP. Unfortunately, in this paper, we reveal that the fairness metric Delta DP can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value Delta DP does not guarantee zero violation of demographic parity, ii) Delta DP values can vary with different classification thresholds. To this end, we propose two new fairness metrics, Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), to precisely measure the violation of demographic parity at the distribution level. The new fairness metrics directly measure the difference between the distributions of the prediction probability for different demographic groups. Thus our proposed new metrics enjoy: i) zero-value ABCC/ABPC guarantees zero violation of demographic parity; ii) ABCC/ABPC guarantees demographic parity while the classification thresholds are adjusted. We further re-evaluate the existing fair models with our proposed fairness metrics and observe different fairness behaviors of those models under the new metrics. The code is available at https://github.com/ahxt/newmetricfordemographicparity