# Evaluation of post-hoc interpretability methods in time-series classification

arXiv: https://arxiv.org/abs/2202.05656 · HTML: https://mlsanity.com/paper/2202.05656/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 57 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 28 of 28)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [The UCR Time Series Archive](https://mlsanity.com/paper/1810.07758/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere](https://mlsanity.com/paper/2003.11927/) (2020)
- [Voice2Series: Reprogramming Acoustic Models for Time Series Classification](https://mlsanity.com/paper/2106.09296/) (2021)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets](https://mlsanity.com/paper/1905.10854/) (2019)
- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [Feature Space Transfer for Data Augmentation](https://mlsanity.com/paper/1801.04356/) (2018)
- [Deep Learning for Road Traffic Forecasting: Does it Make a Difference?](https://mlsanity.com/paper/2012.02260/) (2020)
- [Medical Imaging and Machine Learning](https://mlsanity.com/paper/2103.01938/) (2021)

## Built on by (most cited first, 30 of 53)

- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [A Comprehensive Review on Financial Explainable AI](https://mlsanity.com/paper/2309.11960/) (2023)
- [Explainable Natural Language Processing for Corporate Sustainability Analysis](https://mlsanity.com/paper/2407.17487/) (2024)
- [Large Language Models in Plant Biology](https://mlsanity.com/paper/2401.02789/) (2024)
- [ProtoS-ViT: Visual foundation models for sparse self-explainable classifications](https://mlsanity.com/paper/2406.10025/) (2024)
- [Interpretable machine learning for time-to-event prediction in medicine and healthcare](https://mlsanity.com/paper/2303.09817/) (2023)
- [Sparse learned kernels for interpretable and efficient medical time series processing](https://mlsanity.com/paper/2307.05385/) (2023)
- [Enhancing Interpretability and Generalizability in Extended Isolation Forests](https://mlsanity.com/paper/2310.05468/) (2023)
- [Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence](https://mlsanity.com/paper/2406.10557/) (2024)
- [Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification](https://mlsanity.com/paper/2406.12507/) (2024)
- [Revisiting the robustness of post-hoc interpretability methods](https://mlsanity.com/paper/2407.19683/) (2024)
- [Class-Dependent Perturbation Effects in Evaluating Time Series Attributions](https://mlsanity.com/paper/2502.17022/) (2025)
- [Don't Get Me Wrong: How to Apply Deep Visual Interpretations to Time Series](https://mlsanity.com/paper/2203.07861/) (2022)
- [What Sketch Explainability Really Means for Downstream Tasks](https://mlsanity.com/paper/2403.09480/) (2024)
- [CAFO: Feature-Centric Explanation on Time Series Classification](https://mlsanity.com/paper/2406.01833/) (2024)
- [ProtoECGNet: Case-Based Interpretable Deep Learning for Multi-Label ECG Classification with Contrastive Learning](https://mlsanity.com/paper/2504.08713/) (2025)
- [Explainable AI: Learning from the Learners](https://mlsanity.com/paper/2601.05525/) (2026)
- [XAI4Extremes: An interpretable machine learning framework for understanding extreme-weather precursors under climate change](https://mlsanity.com/paper/2503.08163/) (2025)
- [Transparent AI: The Case for Interpretability and Explainability](https://mlsanity.com/paper/2507.23535/) (2025)
- [Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities](https://mlsanity.com/paper/2308.08407/) (2023)
- [Evaluating Explanation Methods for Multivariate Time Series Classification](https://mlsanity.com/paper/2308.15223/) (2023)
- [Quantum Adjoint Convolutional Layers for Effective Data Representation](https://mlsanity.com/paper/2404.17378/) (2024)
- [MetaWearS: A Shortcut in Wearable Systems Lifecycle with Only a Few Shots](https://mlsanity.com/paper/2408.01988/) (2024)
- [Tell me why: Visual foundation models as self-explainable classifiers](https://mlsanity.com/paper/2502.19577/) (2025)
- [Why Do Class-Dependent Evaluation Effects Occur with Time Series Feature Attributions? A Synthetic Data Investigation](https://mlsanity.com/paper/2506.11790/) (2025)
- [AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE](https://mlsanity.com/paper/2606.03631/) (2026)
- [FinXABSA: Explainable Finance through Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2303.02563/) (2023)
- [Exploring Neural Granger Causality with xLSTMs: Unveiling Temporal Dependencies in Complex Data](https://mlsanity.com/paper/2502.09981/) (2025)
- [Beyond Explainability: The Case for AI Validation](https://mlsanity.com/paper/2505.21570/) (2025)
- [Towards Interpretable Concept Learning over Time Series via Temporal Logic Semantics](https://mlsanity.com/paper/2508.03269/) (2025)

## Abstract (arXiv)

Post-hoc interpretability methods are critical tools to explain neural-network results. Several post-hoc methods have emerged in recent years, but when applied to a given task, they produce different results, raising the question of which method is the most suitable to provide correct post-hoc interpretability. To understand the performance of each method, quantitative evaluation of interpretability methods is essential. However, currently available frameworks have several drawbacks which hinders the adoption of post-hoc interpretability methods, especially in high-risk sectors. In this work, we propose a framework with quantitative metrics to assess the performance of existing post-hoc interpretability methods in particular in time series classification. We show that several drawbacks identified in the literature are addressed, namely dependence on human judgement, retraining, and shift in the data distribution when occluding samples. We additionally design a synthetic dataset with known discriminative features and tunable complexity. The proposed methodology and quantitative metrics can be used to understand the reliability of interpretability methods results obtained in practical applications. In turn, they can be embedded within operational workflows in critical fields that require accurate interpretability results for e.g., regulatory policies.