# TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models

arXiv: https://arxiv.org/abs/2109.08438 · HTML: https://mlsanity.com/paper/2109.08438/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 17 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning](https://mlsanity.com/paper/1908.00087/) (2019)
- [Towards a Rigorous Evaluation of XAI Methods on Time Series](https://mlsanity.com/paper/1909.07082/) (2019)
- [Clinical Intervention Prediction and Understanding using Deep Networks](https://mlsanity.com/paper/1705.08498/) (2017)
- [TSViz: Demystification of Deep Learning Models for Time-Series Analysis](https://mlsanity.com/paper/1802.02952/) (2018)
- [timeXplain -- A Framework for Explaining the Predictions of Time Series Classifiers](https://mlsanity.com/paper/2007.07606/) (2020)
- [An Empirical Study of Explainable AI Techniques on Deep Learning Models For Time Series Tasks](https://mlsanity.com/paper/2012.04344/) (2020)

## Built on by (most cited first, 28 of 28)

- [A Survey of Explainable Artificial Intelligence (XAI) in Financial Time Series Forecasting](https://mlsanity.com/paper/2407.15909/) (2024)
- [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022)
- [Motif-guided Time Series Counterfactual Explanations](https://mlsanity.com/paper/2211.04411/) (2022)
- [A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI](https://mlsanity.com/paper/2307.05104/) (2023)
- [Counterfactual Explanations for Time Series Forecasting](https://mlsanity.com/paper/2310.08137/) (2023)
- [Time Series Model Attribution Visualizations as Explanations](https://mlsanity.com/paper/2109.12935/) (2021)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [Class-Specific Explainability for Deep Time Series Classifiers](https://mlsanity.com/paper/2210.05411/) (2022)
- [The FIX Benchmark: Extracting Features Interpretable to eXperts](https://mlsanity.com/paper/2409.13684/) (2024)
- [Explaining Large Language Models with gSMILE](https://mlsanity.com/paper/2505.21657/) (2025)
- [Introducing the Attribution Stability Indicator: a Measure for Time Series XAI Attributions](https://mlsanity.com/paper/2310.04178/) (2023)
- [Mapping the Mind of an Instruction-based Image Editing using SMILE](https://mlsanity.com/paper/2412.16277/) (2024)
- [PAX-TS: Model-agnostic multi-granular explanations for time series forecasting via localized perturbations](https://mlsanity.com/paper/2508.18982/) (2025)
- [Developing Guidelines for Functionally-Grounded Evaluation of Explainable Artificial Intelligence using Tabular Data](https://mlsanity.com/paper/2410.12803/) (2024)
- [Using a Local Surrogate Model to Interpret Temporal Shifts in Global Annual Data](https://mlsanity.com/paper/2404.11874/) (2024)
- [Explaining Clustering of Ecological Momentary Assessment Data Through Temporal and Feature Attention](https://mlsanity.com/paper/2405.04854/) (2024)
- [Explainable Deep Learning Framework for Human Activity Recognition](https://mlsanity.com/paper/2408.11552/) (2024)
- [Surrogate Modeling for Explainable Predictive Time Series Corrections](https://mlsanity.com/paper/2412.19897/) (2024)
- [On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating](https://mlsanity.com/paper/2508.20437/) (2025)
- [ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI](https://mlsanity.com/paper/2607.09649/) (2026)
- [TSFeatLIME: An Online User Study in Enhancing Explainability in Univariate Time Series Forecasting](https://mlsanity.com/paper/2409.15950/) (2024)
- [EVolutionary Independent DEtermiNistiC Explanation](https://mlsanity.com/paper/2501.16357/) (2025)
- [Evaluating Simplification Algorithms for Interpretability of Time Series Classification](https://mlsanity.com/paper/2505.08846/) (2025)
- [Robust Multi-Modal Forecasting: Integrating Static and Dynamic Features](https://mlsanity.com/paper/2505.15083/) (2025)
- [MUPAX: Multidimensional Problem Agnostic eXplainable AI](https://mlsanity.com/paper/2507.13090/) (2025)
- [Interpretability in Deep Time Series Models Demands Semantic Alignment](https://mlsanity.com/paper/2602.02239/) (2026)
- [L2GTX: From Local to Global Time Series Explanations](https://mlsanity.com/paper/2603.13065/) (2026)
- [Feature Attribution-Based Explainability Analysis of Deep Learning Models in Predictive Process Monitoring](https://mlsanity.com/paper/2607.17783/) (2026)

## Abstract (arXiv)

Time series forecasting is a demanding task ranging from weather to failure forecasting with black-box models achieving state-of-the-art performances. However, understanding and debugging are not guaranteed. We propose TS-MULE, a local surrogate model explanation method specialized for time series extending the LIME approach. Our extended LIME works with various ways to segment and perturb the time series data. In our extension, we present six sampling segmentation approaches for time series to improve the quality of surrogate attributions and demonstrate their performances on three deep learning model architectures and three common multivariate time series datasets.