# Learning to Estimate Shapley Values with Vision Transformers

arXiv: https://arxiv.org/abs/2206.05282 · HTML: https://mlsanity.com/paper/2206.05282/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 80 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)

## Built on by (most cited first, 30 of 45)

- [What does a platypus look like? Generating customized prompts for zero-shot image classification](https://mlsanity.com/paper/2209.03320/) (2022)
- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus](https://mlsanity.com/paper/2302.07265/) (2023)
- [Shifting AI Efficiency From Model-Centric to Data-Centric Compression](https://mlsanity.com/paper/2505.19147/) (2025)
- [ViT-CX: Causal Explanation of Vision Transformers](https://mlsanity.com/paper/2211.03064/) (2022)
- [Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution](https://mlsanity.com/paper/2401.15866/) (2024)
- [CoRTX: Contrastive Framework for Real-time Explanation](https://mlsanity.com/paper/2303.02794/) (2023)
- [On the Robustness of Removal-Based Feature Attributions](https://mlsanity.com/paper/2306.07462/) (2023)
- [Efficient XAI Techniques: A Taxonomic Survey](https://mlsanity.com/paper/2302.03225/) (2023)
- [Locality Alignment Improves Vision-Language Models](https://mlsanity.com/paper/2410.11087/) (2024)
- [Explainability of Vision Transformers: A Comprehensive Review and New Perspectives](https://mlsanity.com/paper/2311.06786/) (2023)
- [Interpretability Needs a New Paradigm](https://mlsanity.com/paper/2405.05386/) (2024)
- [Rethinking Explainable Machine Learning as Applied Statistics](https://mlsanity.com/paper/2402.02870/) (2024)
- [SHAP-XRT: The Shapley Value Meets Conditional Independence Testing](https://mlsanity.com/paper/2207.07038/) (2022)
- [Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups](https://mlsanity.com/paper/2310.16316/) (2023)
- [Identifying Important Group of Pixels using Interactions](https://mlsanity.com/paper/2401.03785/) (2024)
- [Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models](https://mlsanity.com/paper/2406.12649/) (2024)
- [SHAP@k:Efficient and Probably Approximately Correct (PAC) Identification of Top-k Features](https://mlsanity.com/paper/2307.04850/) (2023)
- [LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributions](https://mlsanity.com/paper/2411.16760/) (2024)
- [The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research](https://mlsanity.com/paper/2310.07882/) (2023)
- [SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queries](https://mlsanity.com/paper/2410.19236/) (2024)
- [TreeGrad-Ranker: Feature Ranking via O(L)-Time Gradients for Decision Trees](https://mlsanity.com/paper/2602.11623/) (2026)
- [Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformers](https://mlsanity.com/paper/2410.21815/) (2024)
- [FW-Shapley: Real-time Estimation of Weighted Shapley Values](https://mlsanity.com/paper/2503.06602/) (2025)
- [Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs](https://mlsanity.com/paper/2506.00577/) (2025)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [Enhancing Interpretability for Vision Models via Shapley Value Optimization](https://mlsanity.com/paper/2512.14354/) (2025)
- [Weakly Supervised Learning Significantly Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CT](https://mlsanity.com/paper/2211.15924/) (2022)
- [TVE: Learning Meta-attribution for Transferable Vision Explainer](https://mlsanity.com/paper/2312.15359/) (2023)
- [Prospector Heads: Generalized Feature Attribution for Large Models & Data](https://mlsanity.com/paper/2402.11729/) (2024)

## Abstract (arXiv)

Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations than existing methods for ViTs.