mlsanityData to 14 Oct 2026

Learning to Estimate Shapley Values with Vision Transformers

arXiv 2206.05282 ↗First posted 10 Jun 2022cs.CV cs.LGcited by 45

Not on the map. Why?

Abstract (arXiv)

Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations than existing methods for ViTs.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 49 of its 80 references that are in the map. First posted on arXiv 10 Jun 2022.

Builds on

49 of 80 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  9. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  10. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2017Axiomatic Attribution for Deep Networkscited by 3,384
37 more
  1. 2021Convex optimizationcited by 2,515
  2. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  3. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  6. 2018Sanity Checks for Saliency Mapscited by 1,156
  7. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  8. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  9. 2019Attention is not Explanationcited by 929
  10. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  11. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  12. 2020Quantifying Attention Flow in Transformerscited by 775
  13. 2019Attention is not not Explanationcited by 636
  14. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  15. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  16. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  17. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  18. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  19. 2021Intriguing Properties of Vision Transformerscited by 446
  20. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  21. 2019Is Attention Interpretable?cited by 389
  22. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 297
  23. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  24. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  25. 2020fastai: A Layered API for Deep Learningcited by 183
  26. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 154
  27. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  28. 2020Neuron Shapley: Discovering the Responsible Neuronscited by 98
  29. 2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
  30. 2022Better plain ViT baselines for ImageNet-1kcited by 98
  31. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 88
  32. 2020Shapley explainability on the data manifoldcited by 88
  33. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  34. 2020Attribution in Scale and Spacecited by 61
  35. 2021Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretationscited by 59
  36. 2022Missingness Bias in Model Debuggingcited by 38
  37. 2021Attention Flows are Shapley Value Explanationscited by 30

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 12; 2025: 7; 2026: 11.

45 papers.

Sort:

By area:

  1. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 267
  2. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  3. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  4. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  5. 2022ViT-CX: Causal Explanation of Vision Transformerscited by 28
  6. 2024Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attributioncited by 21
  7. 2023CoRTX: Contrastive Framework for Real-time Explanationcited by 20
  8. 2023On the Robustness of Removal-Based Feature Attributionscited by 19
  9. 2023Efficient XAI Techniques: A Taxonomic Surveycited by 14
  10. 2024Locality Alignment Improves Vision-Language Modelscited by 13
  11. 2023Explainability of Vision Transformers: A Comprehensive Review and New Perspectivescited by 12
  12. 2024Interpretability Needs a New Paradigmcited by 11
33 more
  1. 2024Rethinking Explainable Machine Learning as Applied Statisticscited by 8
  2. 2022SHAP-XRT: The Shapley Value Meets Conditional Independence Testingcited by 6
  3. 2023Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groupscited by 6
  4. 2024Identifying Important Group of Pixels using Interactionscited by 6
  5. 2024Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Modelscited by 6
  6. 2023SHAP@k:Efficient and Probably Approximately Correct (PAC) Identification of Top-k Featurescited by 5
  7. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  8. 2023The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Researchcited by 4
  9. 2024SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queriescited by 4
  10. 2026TreeGrad-Ranker: Feature Ranking via O(L)-Time Gradients for Decision Treescited by 4
  11. 2024Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformerscited by 3
  12. 2025FW-Shapley: Real-time Estimation of Weighted Shapley Valuescited by 2
  13. 2025Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMscited by 2
  14. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  15. 2025Enhancing Interpretability for Vision Models via Shapley Value Optimizationcited by 2
  16. 2022Weakly Supervised Learning Significantly Reduces the Number of Labels Required for Intracranial Hemorrhage Detection on Head CTcited by 1
  17. 2023TVE: Learning Meta-attribution for Transferable Vision Explainercited by 1
  18. 2024Prospector Heads: Generalized Feature Attribution for Large Models & Datacited by 1
  19. 2024BONES: a Benchmark fOr Neural Estimation of Shapley valuescited by 1
  20. 2024Output Scouting: Auditing Large Language Models for Catastrophic Responsescited by 1
  21. 2025CE-FAM: Concept-Based Explanation via Fusion of Activation Mapscited by 1
  22. 2026From independent patches to coordinated attention: Controlling information flow in vision transformerscited by 1
  23. 2023A Unified Concept-Based System for Local, Global, and Misclassification Explanationscited by 0
  24. 2025Transferring Visual Explainability of Self-Explaining Models to Prediction-Only Models without Additional Trainingcited by 0
  25. 2026What Helps---and What Hurts: Bidirectional Explanations for Vision Transformerscited by 0
  26. 2026DINO-QPM: Adapting Visual Foundation Models for Globally Interpretable Image Classificationcited by 0
  27. 2026XtrAIn: Training-Guided Occlusion for Feature Attributioncited by 0
  28. 2026OperatorSHAP: Fast and Accurate Shapley Value Estimation for Neural Operatorscited by 0
  29. 2026Nonlinear Axiomatic Attribution for Cooperative Gamescited by 0
  30. 2026Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformerscited by 0
  31. 2026SAE-Xplainers: Rule-Based Feature Interpretation for Extreme Earth Eventscited by 0
  32. 2026A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systemscited by 0
  33. 2026Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusioncited by 0