mlsanityData to 14 Oct 2026

Neuron Shapley: Discovering the Responsible Neurons

arXiv 2002.09815 ↗First posted 23 Feb 2020stat.ML cs.CV cs.LGcited by 98

Not on the map. Why?

Abstract (arXiv)

We develop Neuron Shapley as a new framework to quantify the contribution of individual neurons to the prediction and performance of a deep network. By accounting for interactions across neurons, Neuron Shapley is more effective in identifying important filters compared to common approaches based on activation patterns. Interestingly, removing just 30 filters with the highest Shapley scores effectively destroys the prediction accuracy of Inception-v3 on ImageNet. Visualization of these few critical filters provides insights into how the network functions. Neuron Shapley is a flexible framework and can be applied to identify responsible neurons in many tasks. We illustrate additional applications of identifying filters that are responsible for biased prediction in facial recognition and filters that are vulnerable to adversarial attacks. Removing these filters is a quick way to repair models. Enabling all these applications is a new multi-arm bandit algorithm that we developed to efficiently estimate Neuron Shapley values.

Key facts. Cited by 98 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 19; 2023: 11; 2024: 23; 2025: 23; 2026: 8.

98 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 235
  3. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 165
  4. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  5. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  6. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  7. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  8. 2022The Shapley Value in Machine Learningcited by 98
  9. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 45
  10. 2023Enhancing multimodal cooperation via sample-level modality valuationcited by 44
  11. 2024Incremental Residual Concept Bottleneck Modelscited by 43
  12. 2021Shapley Explanation Networkscited by 41
38 more
  1. 2021Few-shot Backdoor Defense Using Shapley Estimationcited by 41
  2. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 38
  3. 2023Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Valuecited by 38
  4. 2022WeightedSHAP: analyzing and improving Shapley based feature attributionscited by 36
  5. 2024Investigating Layer Importance in Large Language Modelscited by 36
  6. 2023Approximating the Shapley Value without Marginal Contributionscited by 34
  7. 2023LINe: Out-of-Distribution Detection by Leveraging Important Neuronscited by 34
  8. 2024On-the-fly Modulation for Balanced Multimodal Learningcited by 32
  9. 2021Attention Flows are Shapley Value Explanationscited by 30
  10. 2023Circuit Breaking: Removing Model Behaviors with Targeted Ablationcited by 27
  11. 2021Finding Discriminative Filters for Specific Degradations in Blind Super-Resolutioncited by 26
  12. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 25
  13. 2024Optimal ablation for interpretabilitycited by 25
  14. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  15. 2021The Shapley Value of Classifiers in Ensemble Gamescited by 23
  16. 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
  17. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  18. 2022HINT: Hierarchical Neuron Concept Explainercited by 17
  19. 2024How and where does CLIP process negation?cited by 16
  20. 2021Evaluating the Values of Sources in Transfer Learningcited by 11
  21. 2021Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanationcited by 9
  22. 2022Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformerscited by 9
  23. 2023Stabilizing Estimates of Shapley Values with Control Variatescited by 9
  24. 2026Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuitscited by 9
  25. 2022Differentially Private Shapley Values for Data Evaluationcited by 8
  26. 2023Causal Analysis for Robust Interpretability of Neural Networkscited by 8
  27. 2025Improving Multimodal Learning Balance and Sufficiency through Data Remixingcited by 8
  28. 2026Causality is Key for Interpretability Claims to Generalisecited by 8
  29. 2024Statistical Significance of Feature Importance Rankingscited by 7
  30. 2025Discovering Influential Neuron Path in Vision Transformerscited by 7
  31. 2022Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanationscited by 6
  32. 2022Neural Payoff Machines: Predicting Fair and Stable Payoff Allocations Among Team Memberscited by 6
  33. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6
  34. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  35. 2021Finding High-Value Training Data Subset through Differentiable Convex Programmingcited by 5
  36. 2021Learning Interpretable Models Through Multi-Objective Neural Architecture Searchcited by 5
  37. 2023Efficient Multiuser AI Downloading via Reusable Knowledge Broadcastingcited by 5
  38. 2022Defense against Backdoor Attacks via Identifying and Purifying Bad Neuronscited by 4

50 most cited of 98 All 98 (JSON)