mlsanityData to 7 Oct 2026

Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey

arXiv 2104.00950 ↗First posted 2 Apr 2021cs.LG cs.AIcited by 71

Not on the map. Why?

Abstract (arXiv)

Most of state of the art methods applied on time series consist of deep learning methods that are too complex to be interpreted. This lack of interpretability is a major drawback, as several applications in the real world are critical tasks, such as the medical field or the autonomous driving field. The explainability of models applied on time series has not gather much attention compared to the computer vision or the natural language processing fields. In this paper, we present an overview of existing explainable AI (XAI) methods applied on time series and illustrate the type of explanations they produce. We also provide a reflection on the impact of these explanation methods to provide confidence and trust in the AI systems.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 40 of its 102 references that are in the map. First posted on arXiv 2 Apr 2021.

Builds on

40 of 102 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  9. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  10. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  11. 2016Time Series Classification from Scratch with Deep Neural Networks: A Strong Baselinecited by 358
  12. 2018The UEA multivariate time series classification archive, 2018cited by 330
28 more
  1. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  2. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 197
  3. 2017LSTM Fully Convolutional Networks for Time Series Classificationcited by 135
  4. 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
  5. 2017What Does Explainable AI Really Mean? A New Conceptualization of Perspectivescited by 72
  6. 2019Towards a Rigorous Evaluation of XAI Methods on Time Seriescited by 67
  7. 2017Clinical Intervention Prediction and Understanding using Deep Networkscited by 35
  8. 2020Interpretable Time Series Classification using Linear Models and Multi-resolution Multi-domain Symbolic Representationscited by 31
  9. 2019Explaining Deep Classification of Time-Series Data with Learned Prototypescited by 30
  10. 2019Fooling a Real Car with Adversarial Traffic Signscited by 29
  11. 2018TSViz: Demystification of Deep Learning Models for Time-Series Analysiscited by 25
  12. 2020Post-hoc explanation of black-box classifiers using confident itemsetscited by 21
  13. 2019ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classificationcited by 20
  14. 2018Detecting and interpreting myocardial infarction using fully convolutional neural networkscited by 19
  15. 2019Reliable and Explainable Machine Learning Methods for Accelerated Material Discoverycited by 18
  16. 2019Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networkscited by 18
  17. 2018Interpreting Deep Classifier by Visual Distillation of Dark Knowledgecited by 9
  18. 2018Focusing on What is Relevant: Time-Series Learning and Understanding using Attentioncited by 8
  19. 2020TSInsight: A local-global attribution framework for interpretability in time-series datacited by 8
  20. 2020Explainable Tensorized Neural Ordinary Differential Equations forArbitrary-step Time Series Predictioncited by 8
  21. 2019Explainability and Adversarial Robustness for RNNscited by 7
  22. 2019TSXplain: Demystification of DNN Decisions for Time-Series using Natural Language and Statistical Featurescited by 5
  23. 2020Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodescited by 5
  24. 2020Attention Mechanism for Multivariate Time Series Recurrent Model Interpretability Applied to the Ironmaking Industrycited by 5
  25. 2018Interpretable Time Series Classification using All-Subsequence Learning and Symbolic Representations in Time and Frequency Domainscited by 4
  26. 2020Generalised Interpretable Shapelets for Irregular Time Seriescited by 4
  27. 2020Series Saliency: Temporal Interpretation for Multivariate Time Series Forecastingcited by 4
  28. 2019Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 13; 2024: 21; 2025: 13; 2026: 12.

71 papers.

Sort:

By area:

  1. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  2. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  3. 2023Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistencycited by 42
  4. 2023AutoGluon-TimeSeries: AutoML for Probabilistic Time Series Forecastingcited by 36
  5. 2021Toward Explainable AI for Regression Modelscited by 30
  6. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  7. 2022WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Valuescited by 24
  8. 2024Explaining Time Series via Contrastive and Locally Sparse Perturbationscited by 22
  9. 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
  10. 2023A Comprehensive Review on Financial Explainable AIcited by 10
  11. 2024Explainable Predictive Maintenance: A Survey of Current Methods, Challenges and Opportunitiescited by 10
  12. 2022Explainable Artificial Intelligence (XAI) for Internet of Things: A Surveycited by 8
38 more
  1. 2024Right on Time: Revising Time Series Models by Constraining their Explanationscited by 7
  2. 2025Which LIME should I trust? Concepts, Challenges, and Solutionscited by 7
  3. 2022Vision Paper: Causal Inference for Interpretable and Robust Machine Learning in Mobility Analysiscited by 6
  4. 2023Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanationscited by 6
  5. 2023AutoXPCR: Automated Multi-Objective Model Selection for Time Series Forecastingcited by 6
  6. 2024FLEXtime: Filterbank learning to explain time seriescited by 6
  7. 2022ReX: A Framework for Incorporating Temporal Information in Model-Agnostic Local Explanation Techniquescited by 5
  8. 2022On the Robustness of Explanations of Deep Neural Network Models: A Surveycited by 5
  9. 2024How Human-Centered Explainable AI Interface Are Designed and Evaluated: A Systematic Surveycited by 5
  10. 2026TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systemscited by 5
  11. 2022Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Seriescited by 4
  12. 2022Painting the black box white: experimental findings from applying XAI to an ECG reading settingcited by 4
  13. 2023The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Researchcited by 4
  14. 2024Explaining Time Series Classification Predictions via Causal Attributionscited by 4
  15. 2024FreqRISE: Explaining time series using frequency maskingcited by 4
  16. 2024Multimodal Methods for Analyzing Learning and Training Environments: A Systematic Literature Reviewcited by 4
  17. 2022TSInterpret: A unified framework for time series interpretabilitycited by 3
  18. 2022An Overview of Violence Detection Techniques: Current Challenges and Future Directionscited by 3
  19. 2023Exploring a Gradient-based Explainable AI Technique for Time-Series Data: A Case Study of Assessing Stroke Rehabilitation Exercisescited by 3
  20. 2023XTSC-Bench: Quantitative Benchmarking for Explainers on Time Series Classificationcited by 3
  21. 2024PUPAE: Intuitive and Actionable Explanations for Time Series Anomaliescited by 3
  22. 2024Towards Gradient-based Time-Series Explanations through a SpatioTemporal Attention Networkcited by 3
  23. 2024Multi-SpaCE: Multi-Objective Subsequence-based Sparse Counterfactual Explanations for Multivariate Time Series Classificationcited by 3
  24. 2026Reasoning-Aware Training for Time Series Forecastingcited by 3
  25. 2023XAI for time-series classification leveraging image highlight methodscited by 2
  26. 2024MicroXercise: A Micro-Level Comparative and Explainable System for Remote Physical Therapycited by 2
  27. 2024Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics with Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosiscited by 2
  28. 2025iTFKAN: Interpretable Time Series Forecasting with Kolmogorov-Arnold Networkcited by 2
  29. 2025Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Modelscited by 2
  30. 2025ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecastingcited by 2
  31. 2024Interpreting Time Series Transformer Models and Sensitivity Analysis of Population Age Groups to COVID-19 Infectionscited by 1
  32. 2024Interpretable Short-Term Load Forecasting via Multi-Scale Temporal Decompositioncited by 1
  33. 2024ECATS: Explainable-by-design concept-based anomaly detection for time seriescited by 1
  34. 2024Surrogate Modeling for Explainable Predictive Time Series Correctionscited by 1
  35. 2026GroupSegment-SHAP: Shapley Value Explanations with Group-Segment Players for Multivariate Time Seriescited by 1
  36. 2022A Time Series Approach to Explainability for Neural Nets with Applications to Risk-Management and Fraud Detectioncited by 0
  37. 2023DeLELSTM: Decomposition-based Linear Explainable LSTM to Capture Instantaneous and Long-term Effects in Time Seriescited by 0
  38. 2023Extracting Interpretable Local and Global Representations from Attention on Time Seriescited by 0

50 most cited of 71 All 71 (JSON)