# Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts

arXiv: https://arxiv.org/abs/2201.10295 · HTML: https://mlsanity.com/paper/2201.10295/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 68 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 29 of 29)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [A survey of algorithmic recourse: definitions, formulations, solutions, and prospects](https://mlsanity.com/paper/2010.04050/) (2020)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [Counterfactual Explanations Can Be Manipulated](https://mlsanity.com/paper/2106.02666/) (2021)
- [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020)
- [Demystifying the Draft EU Artificial Intelligence Act](https://mlsanity.com/paper/2107.03721/) (2021)
- [Fairwashing Explanations with Off-Manifold Detergent](https://mlsanity.com/paper/2007.09969/) (2020)
- [Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End](https://mlsanity.com/paper/2011.04917/) (2020)
- [Discrimination in the Age of Algorithms](https://mlsanity.com/paper/1902.03731/) (2019)
- [Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems](https://mlsanity.com/paper/1806.07552/) (2018)
- [Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods](https://mlsanity.com/paper/1910.02065/) (2019)
- [On Locality of Local Explanation Models](https://mlsanity.com/paper/2106.14648/) (2021)
- [On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness](https://mlsanity.com/paper/2108.05379/) (2021)
- [Exploring Explainability: A Definition, a Model, and a Knowledge Catalogue](https://mlsanity.com/paper/2108.03012/) (2021)

## Built on by (most cited first, 30 of 53)

- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Interpretable Geometric Deep Learning via Learnable Randomness Injection](https://mlsanity.com/paper/2210.16966/) (2022)
- [Attribution-based Explanations that Provide Recourse Cannot be Robust](https://mlsanity.com/paper/2205.15834/) (2022)
- [Pixel-Grounded Prototypical Part Networks](https://mlsanity.com/paper/2309.14531/) (2023)
- [Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNs](https://mlsanity.com/paper/2406.15156/) (2024)
- [TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts](https://mlsanity.com/paper/2510.06162/) (2025)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Explainability in AI Policies: A Critical Review of Communications, Reports, Regulations, and Standards in the EU, US, and UK](https://mlsanity.com/paper/2304.11218/) (2023)
- [Unfooling Perturbation-Based Post Hoc Explainers](https://mlsanity.com/paper/2205.14772/) (2022)
- [Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders](https://mlsanity.com/paper/2505.16004/) (2025)
- [Robustness and Cybersecurity in the EU Artificial Intelligence Act](https://mlsanity.com/paper/2502.16184/) (2025)
- [Mind the Gap! Bridging Explainable Artificial Intelligence and Human Understanding with Luhmann's Functional Theory of Communication](https://mlsanity.com/paper/2302.03460/) (2023)
- [A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for Depression](https://mlsanity.com/paper/2311.13852/) (2023)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Auditing Local Explanations is Hard](https://mlsanity.com/paper/2407.13281/) (2024)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Manipulation Risks in Explainable AI: The Implications of the Disagreement Problem](https://mlsanity.com/paper/2306.13885/) (2023)
- [LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs](https://mlsanity.com/paper/2308.01157/) (2023)
- [Why do explanations fail? A typology and discussion on failures in XAI](https://mlsanity.com/paper/2405.13474/) (2024)
- [A Means-End Account of Explainable Artificial Intelligence](https://mlsanity.com/paper/2208.04638/) (2022)
- [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots](https://mlsanity.com/paper/2404.18702/) (2024)
- [Explainable AI needs formalization](https://mlsanity.com/paper/2409.14590/) (2024)
- [Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning](https://mlsanity.com/paper/2603.23854/) (2026)
- [Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models](https://mlsanity.com/paper/2606.14647/) (2026)
- [Disagreement amongst counterfactual explanations: How transparency can be deceptive](https://mlsanity.com/paper/2304.12667/) (2023)
- [On the Relationship Between Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2311.11491/) (2023)
- [CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs](https://mlsanity.com/paper/2509.00971/) (2025)
- [The Case Against Explainability](https://mlsanity.com/paper/2305.12167/) (2023)

## Abstract (arXiv)

Existing and planned legislation stipulates various obligations to provide information about machine learning algorithms and their functioning, often interpreted as obligations to "explain". Many researchers suggest using post-hoc explanation algorithms for this purpose. In this paper, we combine legal, philosophical and technical arguments to show that post-hoc explanation algorithms are unsuitable to achieve the law's objectives. Indeed, most situations where explanations are requested are adversarial, meaning that the explanation provider and receiver have opposing interests and incentives, so that the provider might manipulate the explanation for her own ends. We show that this fundamental conflict cannot be resolved because of the high degree of ambiguity of post-hoc explanations in realistic application scenarios. As a consequence, post-hoc explanation algorithms are unsuitable to achieve the transparency objectives inherent to the legal norms. Instead, there is a need to more explicitly discuss the objectives underlying "explainability" obligations as these can often be better achieved through other mechanisms. There is an urgent need for a more open and honest discussion regarding the potential and limitations of post-hoc explanations in adversarial contexts, in particular in light of the current negotiations of the European Union's draft Artificial Intelligence Act.