# TsSHAP: Robust model agnostic feature-based explainability for time series forecasting

arXiv: https://arxiv.org/abs/2303.12316 · HTML: https://mlsanity.com/paper/2303.12316/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 22 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Evaluating and Aggregating Feature-based Model Explanations](https://mlsanity.com/paper/2005.00631/) (2020)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [Towards a Rigorous Evaluation of Explainability for Multivariate Time Series](https://mlsanity.com/paper/2104.04075/) (2021)
- [LoMEF: A Framework to Produce Local Explanations for Global Model Time Series Forecasts](https://mlsanity.com/paper/2111.07001/) (2021)

## Built on by (most cited first, 5 of 5)

- [Explaining Time Series Classification Predictions via Causal Attributions](https://mlsanity.com/paper/2405.15871/) (2024)
- [Quadrature-TreeSHAP: Depth-Independent TreeSHAP and Shapley Interactions](https://mlsanity.com/paper/2605.04497/) (2026)
- [INSIGHTS: Demonstration-Based Summaries of Time Series Predictors](https://mlsanity.com/paper/2605.18849/) (2026)
- [Autorelevance function and other feature relevance measures for univariate time series](https://mlsanity.com/paper/2607.01959/) (2026)
- [A Hilbert-Valued Functional Decomposition Framework for Explaining Time-Dependent Outputs](https://mlsanity.com/paper/2609.11295/) (2026)

## Abstract (arXiv)

A trustworthy machine learning model should be accurate as well as explainable. Understanding why a model makes a certain decision defines the notion of explainability. While various flavors of explainability have been well-studied in supervised learning paradigms like classification and regression, literature on explainability for time series forecasting is relatively scarce. In this paper, we propose a feature-based explainability algorithm, TsSHAP, that can explain the forecast of any black-box forecasting model. The method is agnostic of the forecasting model and can provide explanations for a forecast in terms of interpretable features defined by the user a prior. The explanations are in terms of the SHAP values obtained by applying the TreeSHAP algorithm on a surrogate model that learns a mapping between the interpretable feature space and the forecast of the black-box model. Moreover, we formalize the notion of local, semi-local, and global explanations in the context of time series forecasting, which can be useful in several scenarios. We validate the efficacy and robustness of TsSHAP through extensive experiments on multiple datasets.