mlsanityData to 7 Oct 2026

Feature Attribution Explanations for Spiking Neural Networks

arXiv 2311.02110 ↗First posted 2 Nov 2023cs.NE cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Third-generation artificial neural networks, Spiking Neural Networks (SNNs), can be efficiently implemented on hardware. Their implementation on neuromorphic chips opens a broad range of applications, such as machine learning-based autonomous control and intelligent biomedical devices. In critical applications, however, insight into the reasoning of SNNs is important, thus SNNs need to be equipped with the ability to explain how decisions are reached. We present textitTemporal Spike Attribution (TSA), a local explanation method for SNNs. To compute the explanation, we aggregate all information available in model-internal variables: spike times and model weights. We evaluate TSA on artificial and real-world time series data and measure explanation quality w.r.t. multiple quantitative criteria. We find that TSA correctly identifies a small subset of input features relevant to the decision (i.e., is output-complete and compact) and generates similar explanations for similar inputs (i.e., is continuous). Further, our experiments show that incorporating the notion of absent spikes improves explanation quality. Our work can serve as a starting point for explainable SNNs, with future implementations on hardware yielding not only predictions but also explanations in a broad range of application scenarios. Source code is available at https://github.com/ElisaNguyen/tsa-explanations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 2 Nov 2023.

Builds on

20 of 35 references.

References · 20

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  4. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  5. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  6. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  7. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  8. 2019On the (In)fidelity and Sensitivity for Explanationscited by 309
  9. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  10. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  11. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 185
  12. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 178
8 more
  1. 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
  2. 2019Towards a Rigorous Evaluation of XAI Methods on Time Seriescited by 67
  3. 2021Synthetic Benchmarks for Scientific Research in Explainable Machine Learningcited by 59
  4. 2019A Comprehensive Analysis on Adversarial Robustness of Spiking Neural Networkscited by 35
  5. 2020Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applicationscited by 34
  6. 2021Visual Explanations from Spiking Neural Networks using Interspike Intervalscited by 26
  7. 2021On the overlooked issue of defining explanation objectives for local-surrogate explainerscited by 5
  8. 2019A novel method for extracting interpretable knowledge from a spiking neural classifier with time-varying synaptic weightscited by 3

Built on by

Not cited yet.