# xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truth

arXiv: https://arxiv.org/abs/2603.06781 · HTML: https://mlsanity.com/paper/2603.06781/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Builds on (most cited first, 19 of 19)

- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond](https://mlsanity.com/paper/2202.06861/) (2022)
- [One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques](https://mlsanity.com/paper/1909.03012/) (2019)
- [Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI](https://mlsanity.com/paper/2003.07258/) (2020)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [Evaluation of post-hoc interpretability methods in time-series classification](https://mlsanity.com/paper/2202.05656/) (2022)
- [Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks](https://mlsanity.com/paper/2401.06654/) (2024)
- [Learning Perturbations to Explain Time Series Predictions](https://mlsanity.com/paper/2305.18840/) (2023)
- [A Deep Dive into Perturbations as Evaluation Technique for Time Series XAI](https://mlsanity.com/paper/2307.05104/) (2023)
- [Robust Explainer Recommendation for Time Series Classification](https://mlsanity.com/paper/2306.05501/) (2023)
- [Evaluating Model Explanations without Ground Truth](https://mlsanity.com/paper/2505.10399/) (2025)
- [Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification](https://mlsanity.com/paper/2406.12507/) (2024)
- [Class-Dependent Perturbation Effects in Evaluating Time Series Attributions](https://mlsanity.com/paper/2502.17022/) (2025)
- [Concept Extraction for Time Series with ECLAD-ts](https://mlsanity.com/paper/2504.05024/) (2025)
- [Why Do Class-Dependent Evaluation Effects Occur with Time Series Feature Attributions? A Synthetic Data Investigation](https://mlsanity.com/paper/2506.11790/) (2025)

## Built on by (most cited first, 1 of 1)

- [Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding](https://mlsanity.com/paper/2603.25251/) (2026)

## Abstract (arXiv)

Evaluating time series attribution methods is difficult because real-world datasets rarely provide ground truth for which time points drive a prediction. A common workaround is to generate synthetic data where class-discriminating features are placed at known locations, but each study currently reimplements this from scratch. We introduce xaitimesynth, a Python package that provides reusable infrastructure for this evaluation approach. The package generates synthetic time series following an additive model where each sample is a sum of background signal and a localized, class-discriminating feature, with the feature window automatically tracked as a ground truth mask. A fluent data generation API and YAML configuration format allow flexible and reproducible dataset definitions for both univariate and multivariate time series. The package also provides standard localization metrics, including AUC-PR, AUC-ROC, Relevance Mass Accuracy, and Relevance Rank Accuracy. xaitimesynth is open source and available at https://github.com/gregorbaer/xaitimesynth.