# Interpretable Data-Based Explanations for Fairness Debugging

arXiv: https://arxiv.org/abs/2112.09745 · HTML: https://mlsanity.com/paper/2112.09745/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 98 references that are in the map.
- First posted on arXiv 17 Dec 2021.

## Builds on (most cited first, 30 of 54)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Machine Unlearning](https://mlsanity.com/paper/1912.03817/) (2019)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Approximate Data Deletion from Machine Learning Models](https://mlsanity.com/paper/2002.10077/) (2020)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Representer Point Selection for Explaining Deep Neural Networks](https://mlsanity.com/paper/1811.09720/) (2018)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Adaptive Machine Unlearning](https://mlsanity.com/paper/2106.04378/) (2021)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers](https://mlsanity.com/paper/1912.03277/) (2019)
- [Stronger Data Poisoning Attacks Break Data Sanitization Defenses](https://mlsanity.com/paper/1811.00741/) (2018)
- [A Symbolic Approach to Explaining Bayesian Network Classifiers](https://mlsanity.com/paper/1805.03364/) (2018)
- [Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability](https://mlsanity.com/paper/1910.06358/) (2019)

## Built on by (most cited first, 30 of 30)

- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations](https://mlsanity.com/paper/2205.03295/) (2022)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models](https://mlsanity.com/paper/2206.03945/) (2022)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Enhancing the Rationale-Input Alignment for Self-explaining Rationalization](https://mlsanity.com/paper/2312.04103/) (2023)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Quantifying and mitigating the impact of label errors on model disparity metrics](https://mlsanity.com/paper/2310.02533/) (2023)
- [SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generation](https://mlsanity.com/paper/2411.01710/) (2024)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [A Catalog of Fairness-Aware Practices in Machine Learning Engineering](https://mlsanity.com/paper/2408.16683/) (2024)
- [On Explaining Unfairness: An Overview](https://mlsanity.com/paper/2402.10762/) (2024)
- [Explainable AI is Responsible AI: How Explainability Creates Trustworthy and Socially Responsible Artificial Intelligence](https://mlsanity.com/paper/2312.01555/) (2023)
- [Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data](https://mlsanity.com/paper/2501.13818/) (2025)
- [CausalPre: Scalable and Effective Data Pre-Processing for Causal Fairness](https://mlsanity.com/paper/2509.15199/) (2025)
- [Revisiting Lightweight Compiler Provenance Recovery on ARM Binaries](https://mlsanity.com/paper/2305.03934/) (2023)
- [Example-based Explanations for Random Forests using Machine Unlearning](https://mlsanity.com/paper/2402.05007/) (2024)
- [Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle](https://mlsanity.com/paper/2404.18736/) (2024)
- [Towards Interactively Improving ML Data Preparation Code via "Shadow Pipelines"](https://mlsanity.com/paper/2404.19591/) (2024)
- [How Video Meetings Change Your Expression](https://mlsanity.com/paper/2406.00955/) (2024)
- [Data Debugging is NP-hard for Classifiers Trained with SGD](https://mlsanity.com/paper/2408.01365/) (2024)
- [Stress-Testing ML Pipelines with Adversarial Data Corruption](https://mlsanity.com/paper/2506.01230/) (2025)
- [Accumulative SGD Influence Estimation for Data Attribution](https://mlsanity.com/paper/2510.26185/) (2025)
- [VISTA: Knowledge-Driven Vessel Trajectory Imputation with Repair Provenance](https://mlsanity.com/paper/2601.06940/) (2026)
- [Semantic-Guided RL for Interpretable Feature Engineering](https://mlsanity.com/paper/2410.02519/) (2024)
- [Data Acquisition for Improving Model Fairness using Reinforcement Learning](https://mlsanity.com/paper/2412.03009/) (2024)
- [MEGG: Replay via Maximally Extreme GGscore in Incremental Learning for Neural Recommendation Models](https://mlsanity.com/paper/2509.07319/) (2025)
- [Fair Data Pre-Processing with Imperfect Attribute Space](https://mlsanity.com/paper/2603.26456/) (2026)
- [Data Bias Mitigation under Coverage Constraints & The Price of Fairness](https://mlsanity.com/paper/2606.20461/) (2026)

## Abstract (arXiv)

A wide variety of fairness metrics and eXplainable Artificial Intelligence (XAI) approaches have been proposed in the literature to identify bias in machine learning models that are used in critical real-life contexts. However, merely reporting on a model's bias, or generating explanations using existing XAI techniques is insufficient to locate and eventually mitigate sources of bias. We introduce Gopher, a system that produces compact, interpretable and causal explanations for bias or unexpected model behavior by identifying coherent subsets of the training data that are root-causes for this behavior. Specifically, we introduce the concept of causal responsibility that quantifies the extent to which intervening on training data by removing or updating subsets of it can resolve the bias. Building on this concept, we develop an efficient approach for generating the top-k patterns that explain model bias that utilizes techniques from the machine learning (ML) community to approximate causal responsibility and uses pruning rules to manage the large search space for patterns. Our experimental evaluation demonstrates the effectiveness of Gopher in generating interpretable explanations for identifying and debugging sources of bias.