# Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreement

arXiv: https://arxiv.org/abs/2211.08943 · HTML: https://mlsanity.com/paper/2211.08943/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 89 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 24 of 24)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI](https://mlsanity.com/paper/2010.07487/) (2020)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges](https://mlsanity.com/paper/2010.09337/) (2020)
- [Neural networks for post-processing ensemble weather forecasts](https://mlsanity.com/paper/1805.09091/) (2018)
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](https://mlsanity.com/paper/2007.04131/) (2020)
- [A Simple and Effective Model-Based Variable Importance Measure](https://mlsanity.com/paper/1805.04755/) (2018)
- [Correlation and variable importance in random forests](https://mlsanity.com/paper/1310.5726/) (2013)
- [Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability](https://mlsanity.com/paper/1904.03867/) (2019)
- [Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Dataset](https://mlsanity.com/paper/2103.10005/) (2021)
- [Grouped Feature Importance and Combined Features Effect Plot](https://mlsanity.com/paper/2104.11688/) (2021)
- [Relative Feature Importance](https://mlsanity.com/paper/2007.08283/) (2020)
- [Tropical and Extratropical Cyclone Detection Using Deep Learning](https://mlsanity.com/paper/2005.09056/) (2020)
- [Using Machine Learning to Calibrate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System](https://mlsanity.com/paper/2012.00679/) (2020)

## Built on by (most cited first, 14 of 14)

- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning](https://mlsanity.com/paper/2306.15786/) (2023)
- [Interpretability Needs a New Paradigm](https://mlsanity.com/paper/2405.05386/) (2024)
- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [Space-scale Exploration of the Poor Reliability of Deep Learning Models: the Case of the Remote Sensing of Rooftop Photovoltaic Systems](https://mlsanity.com/paper/2408.07828/) (2024)
- [Towards Interpretability in Audio and Visual Affective Machine Learning: A Review](https://mlsanity.com/paper/2306.08933/) (2023)
- [One Wave To Explain Them All: A Unifying Perspective On Feature Attribution](https://mlsanity.com/paper/2410.01482/) (2024)
- [Backward Compatibility in Attributive Explanation and Enhanced Model Training Method](https://mlsanity.com/paper/2408.02298/) (2024)
- [Support-Contra Asymmetry in LLM Explanations](https://mlsanity.com/paper/2510.21884/) (2025)
- [Fighting the disagreement in Explainable Machine Learning with consensus](https://mlsanity.com/paper/2307.01288/) (2023)
- [Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models](https://mlsanity.com/paper/2502.07153/) (2025)
- [Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectives](https://mlsanity.com/paper/2503.13793/) (2025)
- [Mapping data literacy trajectories in K-12 education](https://mlsanity.com/paper/2603.28317/) (2026)
- [Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narratives](https://mlsanity.com/paper/2608.11033/) (2026)

## Abstract (arXiv)

With increasing interest in explaining machine learning (ML) models, the first part of this two-part study synthesizes recent research on methods for explaining global and local aspects of ML models. This study distinguishes explainability from interpretability, local from global explainability, and feature importance versus feature relevance. We demonstrate and visualize different explanation methods, how to interpret them, and provide a complete Python package (scikit-explain) to allow future researchers to explore these products. We also highlight the frequent disagreement between explanation methods for feature rankings and feature effects and provide practical advice for dealing with these disagreements. We used ML models developed for severe weather prediction and sub-freezing road surface temperature prediction to generalize the behavior of the different explanation methods. For feature rankings, there is substantially more agreement on the set of top features (e.g., on average, two methods agree on 6 of the top 10 features) than on specific rankings (on average, two methods only agree on the ranks of 2-3 features in the set of top 10 features). On the other hand, two feature effect curves from different methods are in high agreement as long as the phase space is well sampled. Finally, a lesser-known method, tree interpreter, was found comparable to SHAP for feature effects, and with the widespread use of random forests in geosciences and computational ease of tree interpreter, we recommend it be explored in future research.