# Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining

arXiv: https://arxiv.org/abs/2501.15070 · HTML: https://mlsanity.com/paper/2501.15070/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 25 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [A Transformer-based Framework for Multivariate Time Series Representation Learning](https://mlsanity.com/paper/2010.02803/) (2020)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Evaluating and Aggregating Feature-based Model Explanations](https://mlsanity.com/paper/2005.00631/) (2020)
- [SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption](https://mlsanity.com/paper/2106.15147/) (2021)
- [Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression](https://mlsanity.com/paper/2012.01536/) (2020)
- [FastSHAP: Real-Time Shapley Value Estimation](https://mlsanity.com/paper/2107.07436/) (2021)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing](https://mlsanity.com/paper/1611.01942/) (2016)
- [Time Series as Images: Vision Transformer for Irregularly Sampled Time Series](https://mlsanity.com/paper/2303.12799/) (2023)
- [TimeSHAP: Explaining Recurrent Models through Sequence Perturbations](https://mlsanity.com/paper/2012.00073/) (2020)
- [Adversarial Attacks on Time Series](https://mlsanity.com/paper/1902.10755/) (2019)
- [Online Decorrelation of Humidity and Temperature in Chemical Sensors for Continuous Monitoring](https://mlsanity.com/paper/1608.01719/) (2016)
- [Enhancing Eye-Tracking Performance through Multi-Task Learning Transformer](https://mlsanity.com/paper/2408.05837/) (2024)

## Built on by (most cited first, 5 of 5)

- [Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability](https://mlsanity.com/paper/2506.02138/) (2025)
- [ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations](https://mlsanity.com/paper/2601.23068/) (2026)
- [Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment](https://mlsanity.com/paper/2506.01290/) (2025)
- [RecMind: LLM-Enhanced Graph Neural Networks for Personalized Consumer Recommendations](https://mlsanity.com/paper/2509.06286/) (2025)
- [Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning](https://mlsanity.com/paper/2511.11648/) (2025)

## Abstract (arXiv)

In this paper, we propose ShapTST, a framework that enables time-series transformers to efficiently generate Shapley-value-based explanations alongside predictions in a single forward pass. Shapley values are widely used to evaluate the contribution of different time-steps and features in a test sample, and are commonly generated through repeatedly inferring on each sample with different parts of information removed. Therefore, it requires expensive inference-time computations that occur at every request for model explanations. In contrast, our framework unifies the explanation and prediction in training through a novel Shapley-based pre-training design, which eliminates the undesirable test-time computation and replaces it with a single-time pre-training. Moreover, this specialized pre-training benefits the prediction performance by making the transformer model more effectively weigh different features and time-steps in the time-series, particularly improving the robustness against data noise that is common to raw time-series data. We experimentally validated our approach on eight public datasets, where our time-series model achieved competitive results in both classification and regression tasks, while providing Shapley-based explanations similar to those obtained with post-hoc computation. Our work offers an efficient and explainable solution for time-series analysis tasks in the safety-critical applications.