# General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models

arXiv: https://arxiv.org/abs/2007.04131 · HTML: https://mlsanity.com/paper/2007.04131/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2020.

## Built on by (most cited first, 30 of 75)

- [Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology](https://mlsanity.com/paper/2206.01794/) (2022)
- [A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME](https://mlsanity.com/paper/2305.02012/) (2023)
- [How to avoid machine learning pitfalls: a guide for academic researchers](https://mlsanity.com/paper/2108.02497/) (2021)
- [Causal Explanations and XAI](https://mlsanity.com/paper/2201.13169/) (2022)
- [Counterfactual Explanations for Models of Code](https://mlsanity.com/paper/2111.05711/) (2021)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [GAM Changer: Editing Generalized Additive Models with Interactive Visualization](https://mlsanity.com/paper/2112.03245/) (2021)
- [REPID: Regional Effect Plots with implicit Interaction Detection](https://mlsanity.com/paper/2202.07254/) (2022)
- [The role of causality in explainable artificial intelligence](https://mlsanity.com/paper/2309.09901/) (2023)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoML](https://mlsanity.com/paper/2303.08485/) (2023)
- [Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics](https://mlsanity.com/paper/2409.16756/) (2024)
- [Grouped Feature Importance and Combined Features Effect Plot](https://mlsanity.com/paper/2104.11688/) (2021)
- [Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreement](https://mlsanity.com/paper/2211.08943/) (2022)
- [A Guide to Feature Importance Methods for Scientific Inference](https://mlsanity.com/paper/2404.12862/) (2024)
- [TIGTEC : Token Importance Guided TExt Counterfactuals](https://mlsanity.com/paper/2304.12425/) (2023)
- [RHALE: Robust and Heterogeneity-aware Accumulated Local Effects](https://mlsanity.com/paper/2309.11193/) (2023)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models](https://mlsanity.com/paper/2403.07733/) (2024)
- [Interpretable machine learning for time-to-event prediction in medicine and healthcare](https://mlsanity.com/paper/2303.09817/) (2023)
- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [Confident Feature Ranking](https://mlsanity.com/paper/2307.15361/) (2023)
- [Why do explanations fail? A typology and discussion on failures in XAI](https://mlsanity.com/paper/2405.13474/) (2024)
- [Marginal Effects for Non-Linear Prediction Functions](https://mlsanity.com/paper/2201.08837/) (2022)
- [The Blame Problem in Evaluating Local Explanations, and How to Tackle it](https://mlsanity.com/paper/2310.03466/) (2023)
- [Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots](https://mlsanity.com/paper/2404.18702/) (2024)
- [Efficient SAGE Estimation via Causal Structure Learning](https://mlsanity.com/paper/2304.03113/) (2023)
- [PiML Toolbox for Interpretable Machine Learning Model Development and Diagnostics](https://mlsanity.com/paper/2305.04214/) (2023)
- [iPDP: On Partial Dependence Plots in Dynamic Modeling Scenarios](https://mlsanity.com/paper/2306.07775/) (2023)
- [Variable Importance in High-Dimensional Settings Requires Grouping](https://mlsanity.com/paper/2312.10858/) (2023)

## Abstract (arXiv)

An increasing number of model-agnostic interpretation techniques for machine learning (ML) models such as partial dependence plots (PDP), permutation feature importance (PFI) and Shapley values provide insightful model interpretations, but can lead to wrong conclusions if applied incorrectly. We highlight many general pitfalls of ML model interpretation, such as using interpretation techniques in the wrong context, interpreting models that do not generalize well, ignoring feature dependencies, interactions, uncertainty estimates and issues in high-dimensional settings, or making unjustified causal interpretations, and illustrate them with examples. We focus on pitfalls for global methods that describe the average model behavior, but many pitfalls also apply to local methods that explain individual predictions. Our paper addresses ML practitioners by raising awareness of pitfalls and identifying solutions for correct model interpretation, but also addresses ML researchers by discussing open issues for further research.