mlsanityData to 14 Oct 2026

Retiring DeltaDP: New Distribution-Level Metrics for Demographic Parity

arXiv 2301.13443 ↗First posted 31 Jan 2023cs.LG cs.AI cs.CYcited by 4

Not on the map. Why?

Abstract (arXiv)

Demographic parity is the most widely recognized measure of group fairness in machine learning, which ensures equal treatment of different demographic groups. Numerous works aim to achieve demographic parity by pursuing the commonly used metric Delta DP. Unfortunately, in this paper, we reveal that the fairness metric Delta DP can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value Delta DP does not guarantee zero violation of demographic parity, ii) Delta DP values can vary with different classification thresholds. To this end, we propose two new fairness metrics, Area Between Probability density function Curves (ABPC) and Area Between Cumulative density function Curves (ABCC), to precisely measure the violation of demographic parity at the distribution level. The new fairness metrics directly measure the difference between the distributions of the prediction probability for different demographic groups. Thus our proposed new metrics enjoy: i) zero-value ABCC/ABPC guarantees zero violation of demographic parity; ii) ABCC/ABPC guarantees demographic parity while the classification thresholds are adjusted. We further re-evaluate the existing fair models with our proposed fairness metrics and observe different fairness behaviors of those models under the new metrics. The code is available at https://github.com/ahxt/newmetricfordemographicparity

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 38 of its 72 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

38 of 72 references.

References · 38

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  4. 2017Counterfactual Fairnesscited by 804
  5. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 586
  6. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  7. 2017On Fairness and Calibrationcited by 406
  8. 2017Algorithmic decision making and the cost of fairnesscited by 403
  9. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  10. 2019Flexibly Fair Representation Learning by Disentanglementcited by 196
  11. 2015Censoring Representations with an Adversarycited by 192
  12. 2019Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practicescited by 174
26 more
  1. 2020Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Informationcited by 168
  2. 2019Wasserstein Fair Classificationcited by 131
  3. 2021Towards a Unified Framework for Fair and Stable Graph Representation Learningcited by 128
  4. 2021Fair Mixup: Fairness via Interpolationcited by 109
  5. 2020Fair Regression with Wasserstein Barycenterscited by 100
  6. 2019Training individually fair ML models with Sensitive Subspace Robustnesscited by 73
  7. 2021Fairness in Credit Scoring: Assessment, Implementation and Profit Implicationscited by 65
  8. 2016Learning to Pivot with Adversarial Networkscited by 55
  9. 2020Two Simple Ways to Learn Individual Fairness Metrics from Datacited by 54
  10. 2018How Do Fairness Definitions Fare? Examining Public Attitudes Towards Algorithmic Definitions of Fairnesscited by 48
  11. 2020Fairness Metrics: A Comparative Analysiscited by 45
  12. 2019The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metriccited by 41
  13. 2020SenSeI: Sensitive Set Invariance for Enforcing Individual Fairnesscited by 37
  14. 2022FMP: Toward Fair Graph Message Passing against Topology Biascited by 35
  15. 2022Fairness Reprogrammingcited by 33
  16. 2022Transferring Fairness under Distribution Shifts via Fair Consistency Regularizationcited by 21
  17. 2022Counterfactual Fairness with Partially Known Causal Graphcited by 20
  18. 2022Fair Ranking with Noisy Protected Attributescited by 17
  19. 2018From Soft Classifiers to Hard Decisions: How fair can we be?cited by 16
  20. 2020Wasserstein-based fairness interpretability framework for machine learning modelscited by 14
  21. 2022Domain Adaptation meets Individual Fairness. And they get alongcited by 13
  22. 2020Towards Threshold Invariant Fair Classificationcited by 12
  23. 2017The cost of fairness in classificationcited by 9
  24. 2021Fair Tree Classifier using Strong Demographic Paritycited by 9
  25. 2020Statistical Equity: A Fairness Classification Objectivecited by 7
  26. 2021Model-agnostic bias mitigation methods with regressor distribution control for Wasserstein-based fairness metricscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methodscited by 33
  2. 2024Uplift modeling with continuous treatments: A predict-then-optimize approachcited by 4
  3. 2025Decision-centric fairness: Evaluation and optimization for resource allocation problemscited by 2
  4. 2026Balancing Fairness, Privacy, and Accuracy: A Multitask Adversarial Framework for Centralized Data-Driven Systemscited by 0