# What does LIME really see in images?

arXiv: https://arxiv.org/abs/2102.06307 · HTML: https://mlsanity.com/paper/2102.06307/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020)
- [An Analysis of LIME for Text Data](https://mlsanity.com/paper/2010.12487/) (2020)
- [Looking Deeper into Tabular LIME](https://mlsanity.com/paper/2008.11092/) (2020)

## Built on by (most cited first, 21 of 21)

- [GLIME: General, Stable and Local LIME Explanation](https://mlsanity.com/paper/2311.15722/) (2023)
- [The Manifold Hypothesis for Gradient-Based Explanations](https://mlsanity.com/paper/2206.07387/) (2022)
- [Identifying Spurious Correlations and Correcting them with an Explanation-based Learning](https://mlsanity.com/paper/2211.08285/) (2022)
- [Ensembles of Random SHAPs](https://mlsanity.com/paper/2103.03302/) (2021)
- [Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models](https://mlsanity.com/paper/2403.07733/) (2024)
- [An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data](https://mlsanity.com/paper/2106.09111/) (2021)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [Provably Better Explanations with Optimized Aggregation of Feature Attributions](https://mlsanity.com/paper/2406.05090/) (2024)
- [CAM-Based Methods Can See through Walls](https://mlsanity.com/paper/2404.01964/) (2024)
- [Study on the Helpfulness of Explainable Artificial Intelligence](https://mlsanity.com/paper/2410.11896/) (2024)
- [Using Stratified Sampling to Improve LIME Image Explanations](https://mlsanity.com/paper/2403.17742/) (2024)
- [On The Variability of Concept Activation Vectors](https://mlsanity.com/paper/2509.24058/) (2025)
- [On the Robustness of Text Vectorizers](https://mlsanity.com/paper/2303.07203/) (2023)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [How to scale hyperparameters for quickshift image segmentation](https://mlsanity.com/paper/2201.09286/) (2022)
- [Feature Attribution from First Principles](https://mlsanity.com/paper/2505.24729/) (2025)
- [Stop overkilling simple tasks with black-box models and use transparent models instead](https://mlsanity.com/paper/2302.02804/) (2023)
- [Fairness Explainability using Optimal Transport with Applications in Image Classification](https://mlsanity.com/paper/2308.11090/) (2023)
- [SurvBeNIM: The Beran-Based Neural Importance Model for Explaining the Survival Models](https://mlsanity.com/paper/2312.06638/) (2023)
- [MindfulLIME: A Stable Solution for Explanations of Machine Learning Models with Enhanced Localization Precision -- A Medical Image Case Study](https://mlsanity.com/paper/2503.20758/) (2025)
- [Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations](https://mlsanity.com/paper/2607.15482/) (2026)

## Abstract (arXiv)

The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this paper, we study LIME, perhaps one of the most popular. On the theoretical side, we show that when the number of generated examples is large, LIME explanations are concentrated around a limit explanation for which we give an explicit expression. We further this study for elementary shape detectors and linear models. As a consequence of this analysis, we uncover a connection between LIME and integrated gradients, another explanation method. More precisely, the LIME explanations are similar to the sum of integrated gradients over the superpixels used in the preprocessing step of LIME.