# Can Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AI

arXiv: https://arxiv.org/abs/2106.07483 · HTML: https://mlsanity.com/paper/2106.07483/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 14 of 14)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://mlsanity.com/paper/1708.08296/) (2017)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [The Frontiers of Fairness in Machine Learning](https://mlsanity.com/paper/1810.08810/) (2018)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Fairwashing: the risk of rationalization](https://mlsanity.com/paper/1901.09749/) (2019)
- [The relationship between trust in AI and trustworthy machine learning technologies](https://mlsanity.com/paper/1912.00782/) (2019)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges](https://mlsanity.com/paper/1803.07517/) (2018)
- [Racial categories in machine learning](https://mlsanity.com/paper/1811.11668/) (2018)
- [Proposed Guidelines for the Responsible Use of Explainable Machine Learning](https://mlsanity.com/paper/1906.03533/) (2019)

## Built on by (most cited first, 16 of 16)

- [Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations](https://mlsanity.com/paper/2205.07277/) (2022)
- [Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models](https://mlsanity.com/paper/2206.03945/) (2022)
- [Characterizing Large Language Models as Rationalizers of Knowledge-intensive Tasks](https://mlsanity.com/paper/2311.05085/) (2023)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Disentangling and Operationalizing AI Fairness at LinkedIn](https://mlsanity.com/paper/2306.00025/) (2023)
- [Explainable Global Fairness Verification of Tree-Based Classifiers](https://mlsanity.com/paper/2209.13179/) (2022)
- [On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2510.12201/) (2025)
- [Towards a Praxis for Intercultural Ethics in Explainable AI](https://mlsanity.com/paper/2304.11861/) (2023)
- [Explainable AI in Orthopedics: Challenges, Opportunities, and Prospects](https://mlsanity.com/paper/2308.04696/) (2023)
- [Bias Neutralization Framework: Measuring Fairness in Large Language Models with Bias Intelligence Quotient (BiQ)](https://mlsanity.com/paper/2404.18276/) (2024)
- [On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating](https://mlsanity.com/paper/2508.20437/) (2025)
- [An Analysis of the New EU AI Act and A Proposed Standardization Framework for Machine Learning Fairness](https://mlsanity.com/paper/2510.01281/) (2025)
- [Fairness Explainability using Optimal Transport with Applications in Image Classification](https://mlsanity.com/paper/2308.11090/) (2023)
- [Beyond XAI:Obstacles Towards Responsible AI](https://mlsanity.com/paper/2309.03638/) (2023)
- [Is Machine Learning Unsafe and Irresponsible in Social Sciences? Paradoxes and Reconsidering from Recidivism Prediction Tasks](https://mlsanity.com/paper/2311.06537/) (2023)

## Abstract (arXiv)

Many ML models are opaque to humans, producing decisions too complex for humans to easily understand. In response, explainable artificial intelligence (XAI) tools that analyze the inner workings of a model have been created. Despite these tools' strength in translating model behavior, critiques have raised concerns about the impact of XAI tools as a tool for `fairwashing` by misleading users into trusting biased or incorrect models. In this paper, we created a framework for evaluating explainable AI tools with respect to their capabilities for detecting and addressing issues of bias and fairness as well as their capacity to communicate these results to their users clearly. We found that despite their capabilities in simplifying and explaining model behavior, many prominent XAI tools lack features that could be critical in detecting bias. Developers can use our framework to suggest modifications needed in their toolkits to reduce issues likes fairwashing.