mlsanityData to 7 Oct 2026

Learning Important Features Through Propagating Activation Differences

arXiv 1704.02685 ↗First posted 10 Apr 2017cs.CV cs.LG cs.NEcited by 1,454

Not on the map. Why?

Abstract (arXiv)

The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, ICML slides: bit.ly/deeplifticmlslides, ICML talk: https://vimeo.com/238275076, code: http://goo.gl/RM8jvH.

Key facts. Cited by 1,454 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 260; 2022: 258; 2023: 278; 2024: 296; 2025: 198; 2026: 163.

1,454 papers.

Sort:

By area:

  1. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  3. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  4. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  5. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  6. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  7. 2021Trustworthy AI: From Principles to Practicescited by 206
  8. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  9. 2022Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyondcited by 171
  10. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 158
  11. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 156
  12. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 144
38 more
  1. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  2. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  4. 2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
  5. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 123
  6. 2021Explaining in Style: Training a GAN to explain a classifier in StyleSpacecited by 122
  7. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  8. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  9. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  10. 2022Algorithms to estimate Shapley value feature attributionscited by 114
  11. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  12. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  13. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  14. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  15. 2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
  16. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  17. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  18. 2022B-cos Networks: Alignment is All We Need for Interpretabilitycited by 88
  19. 2021The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanationscited by 86
  20. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  21. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  22. 2021Explanations in Autonomous Driving: A Surveycited by 80
  23. 2021ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanationcited by 78
  24. 2022Learning and Evaluating Graph Neural Network Explanations based on Counterfactual and Factual Reasoningcited by 78
  25. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  26. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  27. 2024Dense Reward for Free in Reinforcement Learning from Human Feedbackcited by 77
  28. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  29. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 75
  30. 2021Towards Self-Explainable Graph Neural Networkcited by 75
  31. 2023Deep Learning for Survival Analysis: A Reviewcited by 75
  32. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  33. 2021Explaining Time Series Predictions with Dynamic Maskscited by 72
  34. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 71
  35. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
  36. 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
  37. 2022A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributionscited by 64
  38. 2021GraphSVX: Shapley Value Explanations for Graph Neural Networkscited by 63

50 most cited of 1,454 All 1,454 (JSON)