# FairLay-ML: Intuitive Debugging of Fairness in Data-Driven Social-Critical Software

arXiv: https://arxiv.org/abs/2407.01423 · HTML: https://mlsanity.com/paper/2407.01423/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [Automated Directed Fairness Testing](https://mlsanity.com/paper/1807.00468/) (2018)
- [Fairness-aware Configuration of Machine Learning Libraries](https://mlsanity.com/paper/2202.06196/) (2022)
- [Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networks](https://mlsanity.com/paper/2304.04199/) (2023)
- [FairLay-ML: Intuitive Remedies for Unfairness in Data-Driven Social-Critical Algorithms](https://mlsanity.com/paper/2307.05029/) (2023)

## Built on by (most cited first, 4 of 4)

- [Attention Pruning: Automated Fairness Repair of Language Models via Surrogate Simulated Annealing](https://mlsanity.com/paper/2503.15815/) (2025)
- [Uncovering Discrimination Clusters: Quantifying and Explaining Systematic Fairness Violations](https://mlsanity.com/paper/2512.23769/) (2025)
- [Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs](https://mlsanity.com/paper/2608.26209/) (2026)
- [FairLint-DL: An IDE-Native Tool for Fairness Debugging of Deep Learning Software](https://mlsanity.com/paper/2609.16321/) (2026)

## Abstract (arXiv)

Data-driven software solutions have significantly been used in critical domains with significant socio-economic, legal, and ethical implications. The rapid adoptions of data-driven solutions, however, pose major threats to the trustworthiness of automated decision-support software. A diminished understanding of the solution by the developer and historical/current biases in the data sets are primary challenges. To aid data-driven software developers and end-users, we present FairLay-ML, a debugging tool to test and explain the fairness implications of data-driven solutions. FairLay-ML visualizes the logic of datasets, trained models, and decisions for a given data point. In addition, it trains various models with varying fairness-accuracy trade-offs. Crucially, FairLay-ML incorporates counterfactual fairness testing that finds bugs beyond the development datasets. We conducted two studies through FairLay-ML that allowed us to measure false positives/negatives in prevalent counterfactual testing and understand the human perception of counterfactual test cases in a class survey. FairLay-ML and its benchmarks are publicly available at https://github.com/Pennswood/FairLay-ML. The live version of the tool is available at https://fairlayml-v2.streamlit.app/. We provide a video demo of the tool at https://youtu.be/wNI9UWkywVU?t=133.