# Class-Specific Explainability for Deep Time Series Classifiers

arXiv: https://arxiv.org/abs/2210.05411 · HTML: https://mlsanity.com/paper/2210.05411/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Recurrent Neural Networks for Multivariate Time Series with Missing Values](https://mlsanity.com/paper/1606.01865/) (2016)
- [Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline](https://mlsanity.com/paper/1611.06455/) (2016)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [TimeSHAP: Explaining Recurrent Models through Sequence Perturbations](https://mlsanity.com/paper/2012.00073/) (2020)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models](https://mlsanity.com/paper/2109.08438/) (2021)
- [Deletion and Insertion Tests in Regression Models](https://mlsanity.com/paper/2205.12423/) (2022)
- [Automatic time-series phenotyping using massive feature extraction](https://mlsanity.com/paper/1612.05296/) (2016)
- [timeXplain -- A Framework for Explaining the Predictions of Time Series Classifiers](https://mlsanity.com/paper/2007.07606/) (2020)
- [What went wrong and when? Instance-wise Feature Importance for Time-series Models](https://mlsanity.com/paper/2003.02821/) (2020)
- [Human activity recognition based on time series analysis using U-Net](https://mlsanity.com/paper/1809.08113/) (2018)

## Built on by (most cited first, 6 of 6)

- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting](https://mlsanity.com/paper/2407.15909/) (2024)
- [Explanation Space: A New Perspective into Time Series Interpretability](https://mlsanity.com/paper/2409.01354/) (2024)
- [Benchmarking Counterfactual Interpretability in Deep Learning Models for Time Series Classification](https://mlsanity.com/paper/2408.12666/) (2024)
- [On the Necessity of Multi-Domain Explanation: An Uncertainty Principle Approach for Deep Time Series Models](https://mlsanity.com/paper/2506.03267/) (2025)
- [Feature Attribution-Based Explainability Analysis of Deep Learning Models in Predictive Process Monitoring](https://mlsanity.com/paper/2607.17783/) (2026)

## Abstract (arXiv)

Explainability helps users trust deep learning solutions for time series classification. However, existing explainability methods for multi-class time series classifiers focus on one class at a time, ignoring relationships between the classes. Instead, when a classifier is choosing between many classes, an effective explanation must show what sets the chosen class apart from the rest. We now formalize this notion, studying the open problem of class-specific explainability for deep time series classifiers, a challenging and impactful problem setting. We design a novel explainability method, DEMUX, which learns saliency maps for explaining deep multi-class time series classifiers by adaptively ensuring that its explanation spotlights the regions in an input time series that a model uses specifically to its predicted class. DEMUX adopts a gradient-based approach composed of three interdependent modules that combine to generate consistent, class-specific saliency maps that remain faithful to the classifier's behavior yet are easily understood by end users. Our experimental study demonstrates that DEMUX outperforms nine state-of-the-art alternatives on five popular datasets when explaining two types of deep time series classifiers. Further, through a case study, we demonstrate that DEMUX's explanations indeed highlight what separates the predicted class from the others in the eyes of the classifier. Our code is publicly available at https://github.com/rameshdoddaiah/DEMUX.