# FairX: A comprehensive benchmarking tool for model analysis using fairness, utility, and explainability

arXiv: https://arxiv.org/abs/2406.14281 · HTML: https://mlsanity.com/paper/2406.14281/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks](https://mlsanity.com/paper/2110.12884/) (2021)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [TabFairGAN: Fair Tabular Data Generation with Generative Adversarial Networks](https://mlsanity.com/paper/2109.00666/) (2021)
- [Understanding Instance-Level Impact of Fairness Constraints](https://mlsanity.com/paper/2206.15437/) (2022)
- [TransFusion: Generating Long, High Fidelity Time Series using Diffusion Models with Transformers](https://mlsanity.com/paper/2307.12667/) (2023)
- [Towards Standardizing AI Bias Exploration](https://mlsanity.com/paper/2405.19022/) (2024)
- [Bt-GAN: Generating Fair Synthetic Healthdata via Bias-transforming Generative Adversarial Networks](https://mlsanity.com/paper/2404.13634/) (2024)

## Built on by (most cited first, 2 of 2)

- [Generating Synthetic Fair Syntax-agnostic Data by Learning and Distilling Fair Representation](https://mlsanity.com/paper/2408.10755/) (2024)
- [Fair4Free: Generating High-fidelity Fair Synthetic Samples using Data Free Distillation](https://mlsanity.com/paper/2410.01423/) (2024)

## Abstract (arXiv)

We present FairX, an open-source Python-based benchmarking tool designed for the comprehensive analysis of models under the umbrella of fairness, utility, and eXplainability (XAI). FairX enables users to train benchmarking bias-mitigation models and evaluate their fairness using a wide array of fairness metrics, data utility metrics, and generate explanations for model predictions, all within a unified framework. Existing benchmarking tools do not have the way to evaluate synthetic data generated from fair generative models, also they do not have the support for training fair generative models either. In FairX, we add fair generative models in the collection of our fair-model library (pre-processing, in-processing, post-processing) and evaluation metrics for evaluating the quality of synthetic fair data. This version of FairX supports both tabular and image datasets. It also allows users to provide their own custom datasets. The open-source FairX benchmarking package is publicly available at urlhttps://github.com/fahim-sikder/FairX.