# It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks

arXiv: https://arxiv.org/abs/2106.05498 · HTML: https://mlsanity.com/paper/2106.05498/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2021.

## Built on by (most cited first, 30 of 61)

- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Benchmarking Distribution Shift in Tabular Data with TableShift](https://mlsanity.com/paper/2312.07577/) (2023)
- [Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation](https://mlsanity.com/paper/2502.06559/) (2025)
- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation](https://mlsanity.com/paper/2211.13358/) (2022)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness](https://mlsanity.com/paper/2202.09519/) (2022)
- [Unprocessing Seven Years of Algorithmic Fairness](https://mlsanity.com/paper/2306.07261/) (2023)
- [Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation](https://mlsanity.com/paper/2211.12703/) (2022)
- [Beyond Adult and COMPAS: Fairness in Multi-Class Prediction](https://mlsanity.com/paper/2206.07801/) (2022)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [FRAPPE: A Group Fairness Framework for Post-Processing Everything](https://mlsanity.com/paper/2312.02592/) (2023)
- [On (assessing) the fairness of risk score models](https://mlsanity.com/paper/2302.08851/) (2023)
- [One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions](https://mlsanity.com/paper/2308.16681/) (2023)
- [To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier](https://mlsanity.com/paper/2206.00074/) (2022)
- [DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning](https://mlsanity.com/paper/2210.04389/) (2022)
- [Lazy Data Practices Harm Fairness Research](https://mlsanity.com/paper/2404.17293/) (2024)
- [Fairness for AUC via Feature Augmentation](https://mlsanity.com/paper/2111.12823/) (2021)
- [Gradual (In)Compatibility of Fairness Criteria](https://mlsanity.com/paper/2109.04399/) (2021)
- [Learning Antidote Data to Individual Unfairness](https://mlsanity.com/paper/2211.15897/) (2022)
- [The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning](https://mlsanity.com/paper/2410.09600/) (2024)
- [On the role of benchmarking data sets and simulations in method comparison studies](https://mlsanity.com/paper/2208.01457/) (2022)
- [Cascaded Debiasing: Studying the Cumulative Effect of Multiple Fairness-Enhancing Interventions](https://mlsanity.com/paper/2202.03734/) (2022)
- [Toward Understanding the Disagreement Problem in Neural Network Feature Attribution](https://mlsanity.com/paper/2404.11330/) (2024)
- [Is your model predicting the past?](https://mlsanity.com/paper/2206.11673/) (2022)
- [On the Inductive Biases of Demographic Parity-based Fair Learning Algorithms](https://mlsanity.com/paper/2402.18129/) (2024)
- [The Intersectionality Problem for Algorithmic Fairness](https://mlsanity.com/paper/2411.02569/) (2024)

## Abstract (arXiv)

Risk assessment instrument (RAI) datasets, particularly ProPublica's COMPAS dataset, are commonly used in algorithmic fairness papers due to benchmarking practices of comparing algorithms on datasets used in prior work. In many cases, this data is used as a benchmark to demonstrate good performance without accounting for the complexities of criminal justice (CJ) processes. However, we show that pretrial RAI datasets can contain numerous measurement biases and errors, and due to disparities in discretion and deployment, algorithmic fairness applied to RAI datasets is limited in making claims about real-world outcomes. These reasons make the datasets a poor fit for benchmarking under assumptions of ground truth and real-world impact. Furthermore, conventional practices of simply replicating previous data experiments may implicitly inherit or edify normative positions without explicitly interrogating value-laden assumptions. Without context of how interdisciplinary fields have engaged in CJ research and context of how RAIs operate upstream and downstream, algorithmic fairness practices are misaligned for meaningful contribution in the context of CJ, and would benefit from transparent engagement with normative considerations and values related to fairness, justice, and equality. These factors prompt questions about whether benchmarks for intrinsically socio-technical systems like the CJ system can exist in a beneficial and ethical way.