mlsanityData to 7 Oct 2026

How Important Is a Neuron?

arXiv 1805.12233 ↗First posted 30 May 2018cs.LG stat.MLcited by 73

Not on the map. Why?

Abstract (arXiv)

The problem of attributing a deep network's prediction to its input/base features is well-studied. We introduce the notion of conductance to extend the notion of attribution to the understanding the importance of hidden units. Informally, the conductance of a hidden unit of a deep network is the flow of attribution via this hidden unit. We use conductance to understand the importance of a hidden unit to the prediction for a specific input, or over a set of inputs. We evaluate the effectiveness of conductance in multiple ways, including theoretical properties, ablation studies, and a feature selection task. The empirical evaluations are done using the Inception network over ImageNet data, and a sentiment analysis network over reviews. In both cases, we demonstrate the effectiveness of conductance in identifying interesting insights about the internal workings of these networks.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 16; 2023: 9; 2024: 13; 2025: 8; 2026: 11.

73 papers.

Sort:

By area:

  1. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  2. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  3. 2022Improving Adversarial Transferability via Neuron Attribution-Based Attackscited by 92
  4. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  5. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  6. 2022A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributionscited by 64
  7. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 53
  8. 2024Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMscited by 33
  9. 2021Finding Discriminative Filters for Specific Degradations in Blind Super-Resolutioncited by 26
  10. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  11. 2021Robust Models Are More Interpretable Because Attributions Look Normalcited by 21
  12. 2022Discovering Salient Neurons in Deep NLP Modelscited by 21
38 more
  1. 2024Optimal ablation for interpretabilitycited by 21
  2. 2024Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restorationcited by 20
  3. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  4. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  5. 2021Truly Sparse Neural Networks at Scalecited by 15
  6. 2021MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Datasetcited by 15
  7. 2021Cartoon Explanations of Image Classifierscited by 15
  8. 2022The Manifold Hypothesis for Gradient-Based Explanationscited by 15
  9. 2022Debiasing Deep Chest X-Ray Classifiers using Intra- and Post-processing Methodscited by 15
  10. 2021A Unified Paths Perspective for Pruning at Initializationcited by 14
  11. 2022RoCourseNet: Distributionally Robust Training of a Prediction Aware Recourse Modelcited by 11
  12. 2023NeuroX Library for Neuron Analysis of Deep NLP Modelscited by 11
  13. 2022Continuous Human Action Recognition for Human-Machine Interaction: A Reviewcited by 10
  14. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8
  15. 2025Discovering Influential Neuron Path in Vision Transformerscited by 7
  16. 2022Deeply Explain CNN via Hierarchical Decompositioncited by 6
  17. 2022Adapting to Non-Centered Languages for Zero-shot Multilingual Translationcited by 6
  18. 2023Boosting Adversarial Transferability via Fusing Logits of Top-1 Decomposed Featurecited by 6
  19. 2024SRViT: Vision Transformers for Estimating Radar Reflectivity from Satellite Observations at Scalecited by 6
  20. 2022Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-based Localizationcited by 5
  21. 2022On the Robustness of Explanations of Deep Neural Network Models: A Surveycited by 5
  22. 2023On the special role of class-selective neurons in early trainingcited by 5
  23. 2023Four Axiomatic Characterizations of the Integrated Gradients Attribution Methodcited by 5
  24. 2021Fine-grained Interpretation and Causation Analysis in Deep NLP Modelscited by 4
  25. 2021TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Modelscited by 4
  26. 2021Neuron with Steady Response Leads to Better Generalizationcited by 4
  27. 2023Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generationcited by 4
  28. 2024What Sketch Explainability Really Means for Downstream Taskscited by 4
  29. 2025Explanations of Large Language Models Explain Language Representations in the Braincited by 4
  30. 2022Interpretable Low-Resource Legal Decision Makingcited by 3
  31. 2024PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flipscited by 3
  32. 2021Robust and Interpretable Temporal Convolution Network for Event Detection in Lung Sound Recordingscited by 2
  33. 2025DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Searchcited by 2
  34. 2026Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structurescited by 2
  35. 2026Demystifying Variance in Circuit Discovery of LLMscited by 2
  36. 2021Resilience of Bayesian Layer-Wise Explanations under Adversarial Attackscited by 1
  37. 2022Sparsity-Inducing Categorical Prior Improves Robustness of the Information Bottleneckcited by 1
  38. 2022Quantifying Human Bias and Knowledge to guide ML models during Trainingcited by 1

50 most cited of 73 All 73 (JSON)