# To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods

arXiv: https://arxiv.org/abs/2106.00461 · HTML: https://mlsanity.com/paper/2106.00461/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 33)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [To Trust Or Not To Trust A Classifier](https://mlsanity.com/paper/1805.11783/) (2018)
- [Model-Agnostic Counterfactual Explanations for Consequential Decisions](https://mlsanity.com/paper/1905.11190/) (2019)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems](https://mlsanity.com/paper/1907.09615/) (2019)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [Principles and Practice of Explainable Machine Learning](https://mlsanity.com/paper/2009.11698/) (2020)
- [What Does Explainable AI Really Mean? A New Conceptualization of Perspectives](https://mlsanity.com/paper/1710.00794/) (2017)
- [RuleMatrix: Visualizing and Understanding Classifiers with Rules](https://mlsanity.com/paper/1807.06228/) (2018)
- [Shapley values for feature selection: The good, the bad, and the axioms](https://mlsanity.com/paper/2102.10936/) (2021)
- [Interpretable Credit Application Predictions With Counterfactual Explanations](https://mlsanity.com/paper/1811.05245/) (2018)
- [Explainable Planning](https://mlsanity.com/paper/1709.10256/) (2017)
- [Visual Analytics for Explainable Deep Learning](https://mlsanity.com/paper/1804.02527/) (2018)
- [DALEX: explainers for complex predictive models](https://mlsanity.com/paper/1806.08915/) (2018)
- [Explaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach](https://mlsanity.com/paper/2001.07417/) (2020)
- [FairLens: Auditing Black-box Clinical Decision Support Systems](https://mlsanity.com/paper/2011.04049/) (2020)

## Built on by (most cited first, 26 of 26)

- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [Framework for Evaluating Faithfulness of Local Explanations](https://mlsanity.com/paper/2202.00734/) (2022)
- [Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methods](https://mlsanity.com/paper/2106.09078/) (2021)
- [How Well do Feature Visualizations Support Causal Understanding of CNN Activations?](https://mlsanity.com/paper/2106.12447/) (2021)
- [Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2207.06236/) (2022)
- [Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2307.14239/) (2023)
- [A Comprehensive Review on Financial Explainable AI](https://mlsanity.com/paper/2309.11960/) (2023)
- [s-LIME: Reconciling Locality and Fidelity in Linear Explanations](https://mlsanity.com/paper/2208.01510/) (2022)
- [REVEL Framework to measure Local Linear Explanations for black-box models: Deep Learning Image Classification case of study](https://mlsanity.com/paper/2211.06154/) (2022)
- [Hamming Similarity and Graph Laplacians for Class Partitioning and Adversarial Image Detection](https://mlsanity.com/paper/2305.01808/) (2023)
- [Precise Benchmarking of Explainable AI Attribution Methods](https://mlsanity.com/paper/2308.03161/) (2023)
- [Trust Regions for Explanations via Black-Box Probabilistic Certification](https://mlsanity.com/paper/2402.11168/) (2024)
- [Local Interpretable Model Agnostic Shap Explanations for machine learning models](https://mlsanity.com/paper/2210.04533/) (2022)
- [X-SHIELD: Regularization for eXplainable Artificial Intelligence](https://mlsanity.com/paper/2404.02611/) (2024)
- [RESHAPE: Explaining Accounting Anomalies in Financial Statement Audits by enhancing SHapley Additive exPlanations](https://mlsanity.com/paper/2209.09157/) (2022)
- [Developing Guidelines for Functionally-Grounded Evaluation of Explainable Artificial Intelligence using Tabular Data](https://mlsanity.com/paper/2410.12803/) (2024)
- [Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning](https://mlsanity.com/paper/2201.12143/) (2022)
- [Enriching Artificial Intelligence Explanations with Knowledge Fragments](https://mlsanity.com/paper/2204.05579/) (2022)
- [T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients](https://mlsanity.com/paper/2404.16495/) (2024)
- [Explanations Go Linear: Post-hoc Explainability for Tabular Data with Interpretable Meta-Encoding](https://mlsanity.com/paper/2504.20667/) (2025)
- [Streamlining models with explanations in the learning loop](https://mlsanity.com/paper/2302.07760/) (2023)
- [Using Slisemap to interpret physical data](https://mlsanity.com/paper/2310.15610/) (2023)
- [Slice and Explain: Logic-Based Explanations for Neural Networks through Domain Slicing](https://mlsanity.com/paper/2602.22115/) (2026)
- [Evaluating Local Explainability Metrics for Machine Learning Models on Tabular Data](https://mlsanity.com/paper/2605.27618/) (2026)
- [Explanation-Guided Medical Named Entity Recognition with Stability and Boundary Awareness for Atopic Dermatitis](https://mlsanity.com/paper/2606.22886/) (2026)
- [When Superpixels Fail on Documents: A Study of Segmentation for LIME Explanations](https://mlsanity.com/paper/2609.07462/) (2026)

## Abstract (arXiv)

The main objective of eXplainable Artificial Intelligence (XAI) is to provide effective explanations for black-box classifiers. The existing literature lists many desirable properties for explanations to be useful, but there is no consensus on how to quantitatively evaluate explanations in practice. Moreover, explanations are typically used only to inspect black-box models, and the proactive use of explanations as a decision support is generally overlooked. Among the many approaches to XAI, a widely adopted paradigm is Local Linear Explanations - with LIME and SHAP emerging as state-of-the-art methods. We show that these methods are plagued by many defects including unstable explanations, divergence of actual implementations from the promised theoretical properties, and explanations for the wrong label. This highlights the need to have standard and unbiased evaluation procedures for Local Linear Explanations in the XAI field. In this paper we address the problem of identifying a clear and unambiguous set of metrics for the evaluation of Local Linear Explanations. This set includes both existing and novel metrics defined specifically for this class of explanations. All metrics have been included in an open Python framework, named LEAF. The purpose of LEAF is to provide a reference for end users to evaluate explanations in a standardised and unbiased way, and to guide researchers towards developing improved explainable techniques.