# Time is Not Enough: Time-Frequency based Explanation for Time-Series Black-Box Models

arXiv: https://arxiv.org/abs/2408.03636 · HTML: https://mlsanity.com/paper/2408.03636/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 7 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [LSTM Fully Convolutional Networks for Time Series Classification](https://mlsanity.com/paper/1709.05206/) (2017)
- [Explaining Time Series Predictions with Dynamic Masks](https://mlsanity.com/paper/2106.05303/) (2021)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [Reliable Local Explanations for Machine Listening](https://mlsanity.com/paper/2005.07788/) (2020)
- [A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI](https://mlsanity.com/paper/2307.05104/) (2023)
- [Towards a Rigorous Evaluation of Explainability for Multivariate Time Series](https://mlsanity.com/paper/2104.04075/) (2021)
- [Focusing on What is Relevant: Time-Series Learning and Understanding using Attention](https://mlsanity.com/paper/1806.08523/) (2018)
- [XAI for time-series classification leveraging image highlight methods](https://mlsanity.com/paper/2311.17110/) (2023)

## Built on by (most cited first, 6 of 6)

- [ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue](https://mlsanity.com/paper/2601.20323/) (2026)
- [Into the ORBIT for Time Series: Training Regimes for Foundation Models](https://mlsanity.com/paper/2608.13262/) (2026)
- [Time series saliency maps: explaining models across multiple domains](https://mlsanity.com/paper/2505.13100/) (2025)
- [Implet: A Post-hoc Subsequence Explainer for Time Series Models](https://mlsanity.com/paper/2505.08748/) (2025)
- [CENDRe: Concept Extraction with Natural Domain Representations](https://mlsanity.com/paper/2607.29621/) (2026)
- [Explaining Time Series Forecasting with Horizon-Resolved Attribution](https://mlsanity.com/paper/2609.12639/) (2026)

## Abstract (arXiv)

Despite the massive attention given to time-series explanations due to their extensive applications, a notable limitation in existing approaches is their primary reliance on the time-domain. This overlooks the inherent characteristic of time-series data containing both time and frequency features. In this work, we present Spectral eXplanation (SpectralX), an XAI framework that provides time-frequency explanations for time-series black-box classifiers. This easily adaptable framework enables users to "plug-in" various perturbation-based XAI methods for any pre-trained time-series classification models to assess their impact on the explanation quality without having to modify the framework architecture. Additionally, we introduce Feature Importance Approximations (FIA), a new perturbation-based XAI method. These methods consist of feature insertion, deletion, and combination techniques to enhance computational efficiency and class-specific explanations in time-series classification tasks. We conduct extensive experiments in the generated synthetic dataset and various UCR Time-Series datasets to first compare the explanation performance of FIA and other existing perturbation-based XAI methods in both time-domain and time-frequency domain, and then show the superiority of our FIA in the time-frequency domain with the SpectralX framework. Finally, we conduct a user study to confirm the practicality of our FIA in SpectralX framework for class-specific time-frequency based time-series explanations. The source code is available in https://github.com/gustmd0121/Time_is_not_Enough