Data to 14 Oct 2026

Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods

arXiv 1911.02508 ↗First posted 6 Nov 2019cs.LG cs.AI stat.MLcited by 443

Not on the map. Why?

Abstract (arXiv)

As machine learning black boxes are increasingly being deployed in domains such as healthcare and criminal justice, there is growing emphasis on building tools and techniques for explaining these black boxes in an interpretable manner. Such explanations are being leveraged by domain experts to diagnose systematic errors and underlying biases of black boxes. In this paper, we demonstrate that post hoc explanations techniques that rely on input perturbations, such as LIME and SHAP, are not reliable. Specifically, we propose a novel scaffolding technique that effectively hides the biases of any given classifier by allowing an adversarial entity to craft an arbitrary desired explanation. Our approach can be used to scaffold any biased classifier in such a way that its predictions on the input data distribution still remain biased, but the post hoc explanations of the scaffolded classifier look innocuous. Using extensive evaluation with multiple real-world datasets (including COMPAS), we demonstrate how extremely biased (racist) classifiers crafted by our framework can easily fool popular explanation techniques such as LIME and SHAP into generating innocuous explanations which do not reflect the underlying biases.

Key facts. Cited by 443 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 61; 2022: 97; 2023: 80; 2024: 70; 2025: 68; 2026: 67.

443 papers.

Sort:

By area:

2023Faithful Chain-of-Thought Reasoningcited by 288
2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 178
2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 126
2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
2021Counterfactual Explanations Can Be Manipulatedcited by 113
2023Faithfulness Tests for Natural Language Explanationscited by 107

38 more

2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
2023Can Large Language Models Explain Themselves? A Study of LLM-Generated Self-Explanationscited by 90
2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
2022From Shapley Values to Generalized Additive Models and backcited by 60
2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
2022Neural Basis Models for Interpretabilitycited by 56
2021Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactualscited by 55
2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
2021S-LIME: Stabilized-LIME for Model Explanationcited by 52
2022Rethinking Explainability as a Dialogue: A Practitioner's Perspectivecited by 52
2022Logic-Based Explainability in Machine Learningcited by 48
2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
2022Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Modelscited by 47
2023A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIMEcited by 42
2021Shapley Explanation Networkscited by 41
2022Spurious Features Everywhere -- Large-Scale Detection of Harmful Spurious Features in ImageNetcited by 41
2021SAT-Based Rigorous Explanations for Decision Listscited by 40
2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
2021A Survey on the Robustness of Feature Importance and Counterfactual Explanationscited by 39
2022A Survey on Explainable Anomaly Detectioncited by 39
2021On Efficiently Explaining Graph-Based Classifierscited by 37
2022Can Rationalization Improve Robustness?cited by 36
2021GeCo: Quality Counterfactual Explanations in Real Timecited by 32
2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
2024Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensemblescited by 31
2021Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysiscited by 30
2022Making Heads or Tails: Towards Semantically Consistent Visual Counterfactualscited by 30
2022Sparse Visual Counterfactual Explanations in Image Spacecited by 30
2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
2022Interpretable by Design: Learning Predictors by Composing Interpretable Queriescited by 28
2022SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretabilitycited by 28

50 most cited of 443 All 443 (JSON)