# On Locality of Local Explanation Models

arXiv: https://arxiv.org/abs/2106.14648 · HTML: https://mlsanity.com/paper/2106.14648/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 27 of 27)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [On the Robustness of Interpretability Methods](https://mlsanity.com/paper/1806.08049/) (2018)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Local Rule-Based Explanations of Black Box Decision Systems](https://mlsanity.com/paper/1805.10820/) (2018)
- [Explainable Machine Learning for Scientific Insights and Discoveries](https://mlsanity.com/paper/1905.08883/) (2019)
- [Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression](https://mlsanity.com/paper/2012.01536/) (2020)
- [What do we need to build explainable AI systems for the medical domain?](https://mlsanity.com/paper/1712.09923/) (2017)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability](https://mlsanity.com/paper/1910.06358/) (2019)
- [Model Agnostic Supervised Local Explanations](https://mlsanity.com/paper/1807.02910/) (2018)
- [Evaluations and Methods for Explanation through Robustness Analysis](https://mlsanity.com/paper/2006.00442/) (2020)
- [Measurable Counterfactual Local Explanations for Any Classifier](https://mlsanity.com/paper/1908.03020/) (2019)
- [OptiLIME: Optimized LIME Explanations for Diagnostic Computer Algorithms](https://mlsanity.com/paper/2006.05714/) (2020)
- [Improving LIME Robustness with Smarter Locality Sampling](https://mlsanity.com/paper/2006.12302/) (2020)
- [LimeOut: An Ensemble Approach To Improve Process Fairness](https://mlsanity.com/paper/2006.10531/) (2020)
- [LoRMIkA: Local rule-based model interpretability with k-optimal associations](https://mlsanity.com/paper/1908.03840/) (2019)
- [MeLIME: Meaningful Local Explanation for Machine Learning Models](https://mlsanity.com/paper/2009.05818/) (2020)
- [Local Interpretation Methods to Machine Learning Using the Domain of the Feature Space](https://mlsanity.com/paper/1907.13525/) (2019)

## Built on by (most cited first, 25 of 25)

- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts](https://mlsanity.com/paper/2201.10295/) (2022)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis](https://mlsanity.com/paper/2202.07728/) (2022)
- [Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure](https://mlsanity.com/paper/2206.06219/) (2022)
- [RKHS-SHAP: Shapley Values for Kernel Methods](https://mlsanity.com/paper/2110.09167/) (2021)
- [Explaining the Uncertain: Stochastic Shapley Values for Gaussian Process Models](https://mlsanity.com/paper/2305.15167/) (2023)
- [Unfooling Perturbation-Based Post Hoc Explainers](https://mlsanity.com/paper/2205.14772/) (2022)
- [When and How to Fool Explainable Models (and Humans) with Adversarial Examples](https://mlsanity.com/paper/2107.01943/) (2021)
- [Manifold Restricted Interventional Shapley Values](https://mlsanity.com/paper/2301.04041/) (2023)
- [Explaining Preferences with Shapley Values](https://mlsanity.com/paper/2205.13662/) (2022)
- [Exact Shapley Values for Local and Model-True Explanations of Decision Tree Ensembles](https://mlsanity.com/paper/2112.10592/) (2021)
- [Feature Accentuation: Revealing 'What' Features Respond to in Natural Images](https://mlsanity.com/paper/2402.10039/) (2024)
- [Consistent Sufficient Explanations and Minimal Local Rules for explaining regression and classification models](https://mlsanity.com/paper/2111.04658/) (2021)
- [Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression](https://mlsanity.com/paper/2207.09139/) (2022)
- [Efficient and Accurate Explanation Estimation with Distribution Compression](https://mlsanity.com/paper/2406.18334/) (2024)
- [Pixel-level Certified Explanations via Randomized Smoothing](https://mlsanity.com/paper/2506.15499/) (2025)
- [Attention-like feature explanation for tabular data](https://mlsanity.com/paper/2108.04855/) (2021)
- [Calculating and Visualizing Counterfactual Feature Importance Values](https://mlsanity.com/paper/2306.06506/) (2023)
- [BENK: The Beran Estimator with Neural Kernels for Estimating the Heterogeneous Treatment Effect](https://mlsanity.com/paper/2211.10793/) (2022)
- [Explaining text classifiers through progressive neighborhood approximation with realistic samples](https://mlsanity.com/paper/2302.07733/) (2023)
- [Transparent Neighborhood Approximation for Text Classifier Explanation](https://mlsanity.com/paper/2411.16251/) (2024)
- [Dual feature-based and example-based explanation methods](https://mlsanity.com/paper/2401.16294/) (2024)
- [Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition](https://mlsanity.com/paper/2604.04456/) (2026)
- [Verified SHAP: Provable Bounds for Exact Shapley Values of Neural Networks](https://mlsanity.com/paper/2605.24084/) (2026)

## Abstract (arXiv)

Shapley values provide model agnostic feature attributions for model outcome at a particular instance by simulating feature absence under a global population distribution. The use of a global population can lead to potentially misleading results when local model behaviour is of interest. Hence we consider the formulation of neighbourhood reference distributions that improve the local interpretability of Shapley values. By doing so, we find that the Nadaraya-Watson estimator, a well-studied kernel regressor, can be expressed as a self-normalised importance sampling estimator. Empirically, we observe that Neighbourhood Shapley values identify meaningful sparse feature relevance attributions that provide insight into local model behaviour, complimenting conventional Shapley analysis. They also increase on-manifold explainability and robustness to the construction of adversarial classifiers.