mlsanityData to 14 Oct 2026

Explaining and Interpreting LSTMs

arXiv 1909.12114 ↗First posted 25 Sep 2019cs.LG cs.NE stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explaining the predictions of feed-forward networks to the LSTM architecture used for sequential data modeling and forecasting. The special accumulators and gated interactions present in the LSTM require both a new propagation scheme and an extension of the underlying theoretical framework to deliver faithful explanations.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 7; 2023: 3; 2024: 3; 2025: 3; 2026: 4.

23 papers.

Sort:

By area:

  1. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  2. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 55
  3. 2022On the Explainability of Natural Language Processing Deep Modelscited by 38
  4. 2021Toward Explainable AI for Regression Modelscited by 31
  5. 2022Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspacescited by 25
  6. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 25
  7. 2024The Clever Hans Effect in Unsupervised Learningcited by 24
  8. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  9. 2024Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Featurescited by 7
  10. 2025Recent Advances in Malware Detection: Graph Learning and Explainabilitycited by 7
  11. 2022Explainable Online Lane Change Predictions on a Digital Twin with a Layer Normalized LSTM and Layer-wise Relevance Propagationcited by 5
  12. 2022Implicit N-grams Induced by Recurrencecited by 4
11 more
  1. 2022Learning Temporal Rules from Noisy Timeseries Datacited by 2
  2. 2022NeuroView-RNN: It's About Timecited by 2
  3. 2026Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structurescited by 2
  4. 2023Machine Learning based prediction of Glucose Levels in Type 1 Diabetes Patients with the use of Continuous Glucose Monitoring Datacited by 1
  5. 2021Detection of Dataset Shifts in Learning-Enabled Cyber-Physical Systems using Variational Autoencoder for Regressioncited by 0
  6. 2023Unlocking Layer-wise Relevance Propagation for Autoencoderscited by 0
  7. 2023GAMER-MRIL identifies Disability-Related Brain Changes in Multiple Sclerosiscited by 0
  8. 2024Machine-Learned Closure of URANS for Stably Stratified Turbulence: Connecting Physical Timescales & Data Hyperparameters of Deep Time-Series Modelscited by 0
  9. 2025Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only datacited by 0
  10. 2026Explaining Temporal Graph Neural Networks via Feature-induced Information Flowcited by 0
  11. 2026Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagationcited by 0