mlsanityData to 7 Oct 2026

Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

arXiv 1605.01713 ↗First posted 5 May 2016cs.LG cs.CV cs.NEcited by 295

Not on the map. Why?

Abstract (arXiv)

Note: This paper describes an older version of DeepLIFT. See https://arxiv.org/abs/1704.02685 for the newer version. Original abstract follows: The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. We apply DeepLIFT to models trained on natural images and genomic data, and show significant advantages over gradient-based methods.

Key facts. Cited by 295 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 57; 2022: 55; 2023: 67; 2024: 60; 2025: 34; 2026: 22.

295 papers.

Sort:

By area:

  1. 2021Causal Abstractions of Neural Networkscited by 401
  2. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  3. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  4. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  5. 2024ContextCite: Attributing Model Generation to Contextcited by 93
  6. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  7. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  8. 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
  9. 2022Measuring the Mixing of Contextual Information in the Transformercited by 63
  10. 2022Framework for Evaluating Faithfulness of Local Explanationscited by 55
  11. 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 54
  12. 2021Do Input Gradients Highlight Discriminative Features?cited by 52
38 more
  1. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  2. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 49
  3. 2021Exploiting Explanations for Model Inversion Attackscited by 48
  4. 2021Discretized Integrated Gradients for Explaining Language Modelscited by 45
  5. 2022Learning to Estimate Shapley Values with Vision Transformerscited by 44
  6. 2023Explaining How Transformers Use Context to Build Predictionscited by 42
  7. 2021Shapley Explanation Networkscited by 41
  8. 2023The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantuscited by 37
  9. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  10. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  11. 2021Fast Axiomatic Attribution for Neural Networkscited by 35
  12. 2022Guidelines and Evaluation of Clinical Explainable AI in Medical Image Analysiscited by 34
  13. 2022Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measurecited by 33
  14. 2023Explainable Deep Reinforcement Learning: State of the Art and Challengescited by 33
  15. 2025LLM Pretraining with Continuous Conceptscited by 33
  16. 2022Making Heads or Tails: Towards Semantically Consistent Visual Counterfactualscited by 30
  17. 2022Investigating the fidelity of explainable artificial intelligence methods for applications of convolutional neural networks in geosciencecited by 28
  18. 2021The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Resultscited by 27
  19. 2021Finding Discriminative Filters for Specific Degradations in Blind Super-Resolutioncited by 26
  20. 2021The Role of Explainability in Assuring Safety of Machine Learning in Healthcarecited by 26
  21. 2022Visualizing and Understanding Patch Interactions in Vision Transformercited by 26
  22. 2022Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimizationcited by 25
  23. 2022No Token Left Behind: Explainability-Aided Image Classification and Generationcited by 25
  24. 2022Scalable Interpretability via Polynomialscited by 25
  25. 2022Explainable Intrusion Detection Systems (X-IDS): A Survey of Current Methods, Challenges, and Opportunitiescited by 25
  26. 2023Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Sciencecited by 25
  27. 2021Best of both worlds: local and global explanations with human-understandable conceptscited by 22
  28. 2023Grokking Group Multiplication with Cosetscited by 22
  29. 2022Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?cited by 19
  30. 2024Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformercited by 19
  31. 2024On the Faithfulness of Vision Transformer Explanationscited by 19
  32. 2021Neural Network Attribution Methods for Problems in Geoscience: A Novel Synthetic Benchmark Datasetcited by 18
  33. 2023On the Robustness of Removal-Based Feature Attributionscited by 18
  34. 2023Visual Explanations via Iterated Integrated Attributionscited by 18
  35. 2022HINT: Hierarchical Neuron Concept Explainercited by 17
  36. 2021Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoringcited by 16
  37. 2023HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagationcited by 16
  38. 2021ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNscited by 15

50 most cited of 295 All 295 (JSON)