# Evaluating Fairness in Transaction Fraud Models: Fairness Metrics, Bias Audits, and Challenges

arXiv: https://arxiv.org/abs/2409.04373 · HTML: https://mlsanity.com/paper/2409.04373/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 51 references that are in the map.
- First posted on arXiv 6 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Fairness in Credit Scoring: Assessment, Implementation and Profit Implications](https://mlsanity.com/paper/2103.01907/) (2021)
- [How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations](https://mlsanity.com/paper/2101.08758/) (2021)
- [FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods](https://mlsanity.com/paper/2306.09468/) (2023)
- [Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences](https://mlsanity.com/paper/2401.01641/) (2024)
- [Synthesizing Credit Card Transactions](https://mlsanity.com/paper/1910.03033/) (2019)
- [Understanding Unfairness in Fraud Detection through Model and Data Bias Interactions](https://mlsanity.com/paper/2207.06273/) (2022)

## Built on by (most cited first, 2 of 2)

- [Empirical Analysis of Privacy-Fairness-Accuracy Trade-offs in Federated Learning: A Step Towards Responsible AI](https://mlsanity.com/paper/2503.16233/) (2025)
- [FairMed-XGB: A Bayesian-Optimised Multi-Metric Framework with Explainability for Demographic Equity in Critical Healthcare Data](https://mlsanity.com/paper/2603.14947/) (2026)

## Abstract (arXiv)

Ensuring fairness in transaction fraud detection models is vital due to the potential harms and legal implications of biased decision-making. Despite extensive research on algorithmic fairness, there is a notable gap in the study of bias in fraud detection models, mainly due to the field's unique challenges. These challenges include the need for fairness metrics that account for fraud data's imbalanced nature and the tradeoff between fraud protection and service quality. To address this gap, we present a comprehensive fairness evaluation of transaction fraud models using public synthetic datasets, marking the first algorithmic bias audit in this domain. Our findings reveal three critical insights: (1) Certain fairness metrics expose significant bias only after normalization, highlighting the impact of class imbalance. (2) Bias is significant in both service quality-related parity metrics and fraud protection-related parity metrics. (3) The fairness through unawareness approach, which involved removing sensitive attributes such as gender, does not improve bias mitigation within these datasets, likely due to the presence of correlated proxies. We also discuss socio-technical fairness-related challenges in transaction fraud models. These insights underscore the need for a nuanced approach to fairness in fraud detection, balancing protection and service quality, and moving beyond simple bias mitigation strategies. Future work must focus on refining fairness metrics and developing methods tailored to the unique complexities of the transaction fraud domain.