# Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations

arXiv: https://arxiv.org/abs/2205.07277 · HTML: https://mlsanity.com/paper/2205.07277/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 82 references that are in the map.
- First posted on arXiv 15 May 2022.

## Builds on (most cited first, 30 of 56)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Evaluating and Aggregating Feature-based Model Explanations](https://mlsanity.com/paper/2005.00631/) (2020)
- [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [Model Agnostic Supervised Local Explanations](https://mlsanity.com/paper/1807.02910/) (2018)

## Built on by (most cited first, 30 of 40)

- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanations](https://mlsanity.com/paper/2310.11207/) (2023)
- [Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations](https://mlsanity.com/paper/2206.01254/) (2022)
- ["You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations](https://mlsanity.com/paper/2406.12232/) (2024)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations](https://mlsanity.com/paper/2402.12038/) (2024)
- [Evaluating Model Explanations without Ground Truth](https://mlsanity.com/paper/2505.10399/) (2025)
- [Analyzing Explainer Robustness via Probabilistic Lipschitzness of Prediction Functions](https://mlsanity.com/paper/2206.12481/) (2022)
- [The Disagreement Problem in Faithfulness Metrics](https://mlsanity.com/paper/2311.07763/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Understanding Disparities in Post Hoc Machine Learning Explanation](https://mlsanity.com/paper/2401.14539/) (2024)
- [Minimum Levels of Interpretability for Artificial Moral Agents](https://mlsanity.com/paper/2307.00660/) (2023)
- [GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanations](https://mlsanity.com/paper/2406.11547/) (2024)
- [Trustworthy AI Must Account for Interactions](https://mlsanity.com/paper/2504.07170/) (2025)
- [Survey on AI Ethics: A Socio-technical Perspective](https://mlsanity.com/paper/2311.17228/) (2023)
- [SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learning](https://mlsanity.com/paper/2312.16191/) (2023)
- [Procedural Fairness in Machine Learning](https://mlsanity.com/paper/2404.01877/) (2024)
- [A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysis](https://mlsanity.com/paper/2210.04491/) (2022)
- [On Explaining Unfairness: An Overview](https://mlsanity.com/paper/2402.10762/) (2024)
- [Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods](https://mlsanity.com/paper/2505.01198/) (2025)
- [XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques](https://mlsanity.com/paper/2402.12685/) (2024)
- [A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learning](https://mlsanity.com/paper/2406.00116/) (2024)
- [Iterative Partial Fulfillment of Counterfactual Explanations: Benefits and Risks](https://mlsanity.com/paper/2303.11111/) (2023)
- [T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients](https://mlsanity.com/paper/2404.16495/) (2024)
- [Towards Understanding Sensitive and Decisive Patterns in Explainable AI: A Case Study of Model Interpretation in Geometric Deep Learning](https://mlsanity.com/paper/2407.00849/) (2024)
- [Can LLM Assist in the Evaluation of the Quality of Machine Learning Explanations?](https://mlsanity.com/paper/2502.20635/) (2025)
- [Procedural Fairness via Group Counterfactual Explanation](https://mlsanity.com/paper/2603.11140/) (2026)
- [Explainer Divergence Scores (EDS): Some Post-Hoc Explanations May be Effective for Detecting Unknown Spurious Correlations](https://mlsanity.com/paper/2211.07650/) (2022)
- [Tensions Between the Proxies of Human Values in AI](https://mlsanity.com/paper/2212.07508/) (2022)

## Abstract (arXiv)

As post hoc explanation methods are increasingly being leveraged to explain complex models in high-stakes settings, it becomes critical to ensure that the quality of the resulting explanations is consistently high across various population subgroups including the minority groups. For instance, it should not be the case that explanations associated with instances belonging to a particular gender subgroup (e.g., female) are less accurate than those associated with other genders. However, there is little to no research that assesses if there exist such group-based disparities in the quality of the explanations output by state-of-the-art explanation methods. In this work, we address the aforementioned gaps by initiating the study of identifying group-based disparities in explanation quality. To this end, we first outline the key properties which constitute explanation quality and where disparities can be particularly problematic. We then leverage these properties to propose a novel evaluation framework which can quantitatively measure disparities in the quality of explanations output by state-of-the-art methods. Using this framework, we carry out a rigorous empirical analysis to understand if and when group-based disparities in explanation quality arise. Our results indicate that such disparities are more likely to occur when the models being explained are complex and highly non-linear. In addition, we also observe that certain post hoc explanation methods (e.g., Integrated Gradients, SHAP) are more likely to exhibit the aforementioned disparities. To the best of our knowledge, this work is the first to highlight and study the problem of group-based disparities in explanation quality. In doing so, our work sheds light on previously unexplored ways in which explanation methods may introduce unfairness in real world decision making.