# Towards a Rigorous Evaluation of XAI Methods on Time Series

arXiv: https://arxiv.org/abs/1909.07082 · HTML: https://mlsanity.com/paper/1909.07082/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2019.

## Built on by (most cited first, 30 of 67)

- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey](https://mlsanity.com/paper/2104.00950/) (2021)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Evaluation of post-hoc interpretability methods in time-series classification](https://mlsanity.com/paper/2202.05656/) (2022)
- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models](https://mlsanity.com/paper/2109.08438/) (2021)
- [Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Values](https://mlsanity.com/paper/2110.01307/) (2021)
- [WindowSHAP: An Efficient Framework for Explaining Time-series Classifiers based on Shapley Values](https://mlsanity.com/paper/2211.06507/) (2022)
- [Explainable AI for Time Series via Virtual Inspection Layers](https://mlsanity.com/paper/2303.06365/) (2023)
- [XAIR: A Framework of Explainable AI in Augmented Reality](https://mlsanity.com/paper/2303.16292/) (2023)
- [A Survey on AI Assurance](https://mlsanity.com/paper/2111.07505/) (2021)
- [Interpretation of Time-Series Deep Models: A Survey](https://mlsanity.com/paper/2305.14582/) (2023)
- [Motif-guided Time Series Counterfactual Explanations](https://mlsanity.com/paper/2211.04411/) (2022)
- [A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI](https://mlsanity.com/paper/2307.05104/) (2023)
- [Towards a Rigorous Evaluation of Explainability for Multivariate Time Series](https://mlsanity.com/paper/2104.04075/) (2021)
- [Shapelet-Based Counterfactual Explanations for Multivariate Time Series](https://mlsanity.com/paper/2208.10462/) (2022)
- [Probabilistic Stability Guarantees for Feature Attributions](https://mlsanity.com/paper/2504.13787/) (2025)
- [Time Series Model Attribution Visualizations as Explanations](https://mlsanity.com/paper/2109.12935/) (2021)
- [XAI Methods for Neural Time Series Classification: A Brief Review](https://mlsanity.com/paper/2108.08009/) (2021)
- [Robust Explainer Recommendation for Time Series Classification](https://mlsanity.com/paper/2306.05501/) (2023)
- [Right on Time: Revising Time Series Models by Constraining their Explanations](https://mlsanity.com/paper/2402.12921/) (2024)
- [Class-Specific Explainability for Deep Time Series Classifiers](https://mlsanity.com/paper/2210.05411/) (2022)
- [On the Soundness of XAI in Prognostics and Health Management (PHM)](https://mlsanity.com/paper/2303.05517/) (2023)
- [Shapelet-based Model-agnostic Counterfactual Local Explanations for Time Series Classification](https://mlsanity.com/paper/2402.01343/) (2024)
- [Time is Not Enough: Time-Frequency based Explanation for Time-Series Black-Box Models](https://mlsanity.com/paper/2408.03636/) (2024)
- [FLEXtime: Filterbank learning to explain time series](https://mlsanity.com/paper/2411.05841/) (2024)
- [Applications of Explainable AI for 6G: Technical Aspects, Use Cases, and Research Challenges](https://mlsanity.com/paper/2112.04698/) (2021)
- [ReX: A Framework for Incorporating Temporal Information in Model-Agnostic Local Explanation Techniques](https://mlsanity.com/paper/2209.03798/) (2022)
- [TsSHAP: Robust model agnostic feature-based explainability for time series forecasting](https://mlsanity.com/paper/2303.12316/) (2023)
- [Explaining Exchange Rate Forecasts with Macroeconomic Fundamentals Using Interpretive Machine Learning](https://mlsanity.com/paper/2303.16149/) (2023)

## Abstract (arXiv)

Explainable Artificial Intelligence (XAI) methods are typically deployed to explain and debug black-box machine learning models. However, most proposed XAI methods are black-boxes themselves and designed for images. Thus, they rely on visual interpretability to evaluate and prove explanations. In this work, we apply XAI methods previously used in the image and text-domain on time series. We present a methodology to test and evaluate various XAI methods on time series by introducing new verification techniques to incorporate the temporal dimension. We further conduct preliminary experiments to assess the quality of selected XAI method explanations with various verification methods on a range of datasets and inspecting quality metrics on it. We demonstrate that in our initial experiments, SHAP works robust for all models, but others like DeepLIFT, LRP, and Saliency Maps work better with specific architectures.