mlsanityData to 14 Oct 2026

From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation

arXiv 2206.03208 ↗First posted 7 Jun 2022cs.LG cs.AIcited by 141

Not on the map. Why?

Abstract (arXiv)

The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to today's powerful but opaque deep learning models. While local XAI methods explain individual predictions in form of attribution maps, thereby identifying where important features occur (but not providing information about what they represent), global explanation techniques visualize what concepts a model has generally learned to encode. Both types of methods thus only provide partial insights and leave the burden of interpreting the model's reasoning to the user. In this work we introduce the Concept Relevance Propagation (CRP) approach, which combines the local and global perspectives and thus allows answering both the "where" and "what" questions for individual predictions. We demonstrate the capability of our method in various settings, showcasing that CRP leads to more human interpretable explanations and provides deep insights into the model's representation and reasoning through concept atlases, concept composition analyses, and quantitative investigations of concept subspaces and their role in fine-grained decision making.

Key facts. Cited by 141 papers in the map (data through 14 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

37 of 62 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  6. 2013Intriguing properties of neural networkscited by 5,155
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  11. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,109
25 more
  1. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  2. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  3. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  4. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  5. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  6. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  7. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  8. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  9. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  10. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  11. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 250
  12. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  13. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  14. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  15. 2014Object Detectors Emerge in Deep Scene CNNscited by 179
  16. 2017Learning to Generate Reviews and Discovering Sentimentcited by 148
  17. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  18. 2020Higher-Order Explanations of Graph Neural Networks via Relevant Walkscited by 138
  19. 2017Explaining Recurrent Neural Network Predictions in Sentiment Analysiscited by 88
  20. 2017Learning how to explain neural networks: PatternNet and PatternAttributioncited by 84
  21. 2019Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizationscited by 72
  22. 2019Towards Best Practice in Explaining Neural Network Decisions with LRPcited by 70
  23. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  24. 2016Towards Better Analysis of Deep Convolutional Neural Networkscited by 51
  25. 2021PredDiff: Explanations and Interactions from Conditional Expectationscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 21; 2024: 47; 2025: 49; 2026: 22.

141 papers.

Sort:

By area:

  1. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  2. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  3. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 98
  4. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  5. 2023Labeling Neural Representations with Inverse Recognitioncited by 32
  6. 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
  7. 2025Visual Instance-aware Prompt Tuningcited by 29
  8. 2024Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarkscited by 28
  9. 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
  10. 2024Human-like object concept representations emerge naturally in multimodal large language modelscited by 25
  11. 2023Explainable AI for Time Series via Virtual Inspection Layerscited by 23
  12. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 22
38 more
  1. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 21
  2. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 18
  3. 2024Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlookscited by 18
  4. 2022Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanationscited by 17
  5. 2025Causally Reliable Concept Bottleneck Modelscited by 17
  6. 2023Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discoverycited by 15
  7. 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
  8. 2024WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Conceptscited by 14
  9. 2024Explainable Artificial Intelligence for Medical Applications: A Reviewcited by 12
  10. 2024Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformerscited by 11
  11. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  12. 2025The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentationcited by 11
  13. 2023A Survey on Explainable Artificial Intelligence for Cybersecuritycited by 9
  14. 2024Sui Generis: Large Language Models for Authorship Attribution and Verification in Latincited by 9
  15. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
  16. 2025FACE: Faithful Automatic Concept Extractioncited by 9
  17. 2024Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Featurescited by 7
  18. 2025Tokenize Image Patches: Global Context Fusion for Effective Haze Removal in Large Imagescited by 7
  19. 2024Sparse, Efficient and Explainable Data Attribution with DualXDAcited by 6
  20. 2024How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency mapscited by 6
  21. 2024FLEXtime: Filterbank learning to explain time seriescited by 6
  22. 2025From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliancecited by 6
  23. 2023Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanationscited by 5
  24. 2023Assessment of the Reliablity of a Model's Decision by Generalizing Attribution to the Wavelet Domaincited by 5
  25. 2023Human-Centered Evaluation of XAI Methodscited by 5
  26. 2024Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppressioncited by 5
  27. 2025Representational Similarity via Interpretable Visual Conceptscited by 5
  28. 2025Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representationscited by 5
  29. 2023Glocal Explanations of Expected Goal Models in Soccercited by 4
  30. 2024CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanationscited by 4
  31. 2024Study on the Helpfulness of Explainable Artificial Intelligencecited by 4
  32. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 4
  33. 2023Interpretability in Activation Space Analysis of Transformers: A Focused Surveycited by 3
  34. 2023Towards Interpretability in Audio and Visual Affective Machine Learning: A Reviewcited by 3
  35. 2023Artificial Neural Nets and the Representation of Human Conceptscited by 3
  36. 2024Manipulating Feature Visualizations with Gradient Slingshotscited by 3
  37. 2024Towards Explaining Hypercomplex Neural Networkscited by 3
  38. 2024Fine-Grained Domain Generalization with Feature Structuralizationcited by 3

50 most cited of 141 All 141 (JSON)