mlsanityData to 7 Oct 2026

Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features

arXiv 2408.17198 ↗First posted 30 Aug 2024cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Explainable Artificial Intelligence (XAI) plays a crucial role in fostering transparency and trust in AI systems, where traditional XAI approaches typically offer one level of abstraction for explanations, often in the form of heatmaps highlighting single or multiple input features. However, we ask whether abstract reasoning or problem-solving strategies of a model may also be relevant, as these align more closely with how humans approach solutions to problems. We propose a framework, called Symbolic XAI, that attributes relevance to symbolic queries expressing logical relationships between input features, thereby capturing the abstract reasoning behind a model's predictions. The methodology is built upon a simple yet general multi-order decomposition of model predictions. This decomposition can be specified using higher-order propagation-based relevance methods, such as GNN-LRP, or perturbation-based explanation methods commonly used in XAI. The effectiveness of our framework is demonstrated in the domains of natural language processing (NLP), vision, and quantum chemistry (QC), where abstract symbolic domain knowledge is abundant and of significant interest to users. The Symbolic XAI framework provides an understanding of the model's decision-making process that is both flexible for customization by the user and human-readable through logical formulas.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 74 of its 142 references that are in the map. First posted on arXiv 30 Aug 2024.

Builds on

74 of 142 references.

References · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2018Neural Ordinary Differential Equationscited by 3,733
  12. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
38 more
  1. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  2. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  3. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  4. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  5. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 735
  6. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  7. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  8. 2017SchNet: A continuous-filter convolutional neural network for modeling quantum interactionscited by 569
  9. 2020Directional Message Passing for Molecular Graphscited by 556
  10. 2018Object Detection with Deep Learning: A Reviewcited by 524
  11. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  12. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  13. 2021Equivariant message passing for the prediction of tensorial properties and molecular spectracited by 438
  14. 2020Parameterized Explainer for Graph Neural Networkcited by 423
  15. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
  16. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 383
  17. 2018Invertible Residual Networkscited by 364
  18. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  19. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  20. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  21. 2022Learning Local Equivariant Representations for Large-Scale Atomistic Dynamicscited by 254
  22. 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
  23. 2020Machine Learning Force Fieldscited by 204
  24. 2020Compositional Explanations of Neuronscited by 172
  25. 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 153
  26. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  27. 2019The Shapley Taylor Interaction Indexcited by 143
  28. 2023Getting aligned on representational alignmentcited by 142
  29. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  30. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  31. 2020Higher-Order Explanations of Graph Neural Networks via Relevant Walkscited by 138
  32. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  33. 2020Drug discovery with explainable artificial intelligencecited by 118
  34. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
  35. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  36. 2021SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effectscited by 103
  37. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  38. 2022Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulationscited by 92

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 10
  2. 2026Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathologycited by 2
  3. 2026Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structurescited by 2
  4. 2026Beyond Message Passing: A Symbolic Alternative for Expressive and Interpretable Graph Learningcited by 1
  5. 2025Uncovering the Structure of Explanation Quality with Spectral Analysiscited by 0
  6. 2026Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathologycited by 0
  7. 2026Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directionscited by 0