# Explaining the Explainer: A First Theoretical Analysis of LIME

arXiv: https://arxiv.org/abs/2001.03447 · HTML: https://mlsanity.com/paper/2001.03447/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2020.

## Built on by (most cited first, 30 of 89)

- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [From Shapley Values to Generalized Additive Models and back](https://mlsanity.com/paper/2209.04012/) (2022)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysis](https://mlsanity.com/paper/2106.09992/) (2021)
- [Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline](https://mlsanity.com/paper/2106.06054/) (2021)
- [Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methods](https://mlsanity.com/paper/2106.09078/) (2021)
- [What does LIME really see in images?](https://mlsanity.com/paper/2102.06307/) (2021)
- [Pixel-Grounded Prototypical Part Networks](https://mlsanity.com/paper/2309.14531/) (2023)
- [Connecting Interpretability and Robustness in Decision Trees through Separation](https://mlsanity.com/paper/2102.07048/) (2021)
- [Near-Optimal Explainable k-Means for All Dimensions](https://mlsanity.com/paper/2106.15566/) (2021)
- [Unfooling Perturbation-Based Post Hoc Explainers](https://mlsanity.com/paper/2205.14772/) (2022)
- [Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables](https://mlsanity.com/paper/2306.01464/) (2023)
- [Are Your Explanations Reliable? Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack](https://mlsanity.com/paper/2305.12351/) (2023)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)
- [s-LIME: Reconciling Locality and Fidelity in Linear Explanations](https://mlsanity.com/paper/2208.01510/) (2022)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformers](https://mlsanity.com/paper/2405.13536/) (2024)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [Manipulation Risks in Explainable AI: The Implications of the Disagreement Problem](https://mlsanity.com/paper/2306.13885/) (2023)
- [Explainable Fraud Detection with Deep Symbolic Classification](https://mlsanity.com/paper/2312.00586/) (2023)
- [A Theory of Interpretable Approximations](https://mlsanity.com/paper/2406.10529/) (2024)
- [Certification of embedded systems based on Machine Learning: A survey](https://mlsanity.com/paper/2106.07221/) (2021)
- [An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data](https://mlsanity.com/paper/2106.09111/) (2021)
- [Reducing Unintended Bias of ML Models on Tabular and Textual Data](https://mlsanity.com/paper/2108.02662/) (2021)
- [Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach](https://mlsanity.com/paper/2111.09121/) (2021)
- [A Sea of Words: An In-Depth Analysis of Anchors for Text Data](https://mlsanity.com/paper/2205.13789/) (2022)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [Generative Perturbation Analysis for Probabilistic Black-Box Anomaly Attribution](https://mlsanity.com/paper/2308.04708/) (2023)

## Abstract (arXiv)

Machine learning is used more and more often for sensitive applications, sometimes replacing humans in critical decision-making processes. As such, interpretability of these algorithms is a pressing need. One popular algorithm to provide interpretability is LIME (Local Interpretable Model-Agnostic Explanation). In this paper, we provide the first theoretical analysis of LIME. We derive closed-form expressions for the coefficients of the interpretable model when the function to explain is linear. The good news is that these coefficients are proportional to the gradient of the function to explain: LIME indeed discovers meaningful features. However, our analysis also reveals that poor choices of parameters can lead LIME to miss important features.