# Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency

arXiv: https://arxiv.org/abs/2306.02109 · HTML: https://mlsanity.com/paper/2306.02109/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 112 references that are in the map.
- First posted on arXiv 3 Jun 2023.

## Builds on (most cited first, 30 of 81)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)

## Built on by (most cited first, 30 of 42)

- [UniTS: A Unified Multi-Task Time Series Model](https://mlsanity.com/paper/2403.00131/) (2024)
- [TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop](https://mlsanity.com/paper/2503.01013/) (2025)
- [TimeX++: Learning Time-Series Explanations with Information Bottleneck](https://mlsanity.com/paper/2405.09308/) (2024)
- [Inherently Interpretable Time Series Classification via Multiple Instance Learning](https://mlsanity.com/paper/2311.10049/) (2023)
- [Explaining Time Series via Contrastive and Locally Sparse Perturbations](https://mlsanity.com/paper/2401.08552/) (2024)
- [F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI](https://mlsanity.com/paper/2410.02970/) (2024)
- [TIMING: Temporality-Aware Integrated Gradients for Time Series Explanation](https://mlsanity.com/paper/2506.05035/) (2025)
- [TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models](https://mlsanity.com/paper/2510.01274/) (2025)
- [FTS: A Framework to Find a Faithful TimeSieve](https://mlsanity.com/paper/2405.19647/) (2024)
- [ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models](https://mlsanity.com/paper/2509.22556/) (2025)
- [FLEXtime: Filterbank learning to explain time series](https://mlsanity.com/paper/2411.05841/) (2024)
- [time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models](https://mlsanity.com/paper/2509.05801/) (2025)
- [Adaptive Time Series Reasoning via Segment Selection](https://mlsanity.com/paper/2602.18645/) (2026)
- [Explanation Space: A New Perspective into Time Series Interpretability](https://mlsanity.com/paper/2409.01354/) (2024)
- [ShapeX: Shapelet-Driven Post Hoc Explanations for Time Series Classification Models](https://mlsanity.com/paper/2510.20084/) (2025)
- [Reducing operator complexity in Algebraic Multigrid with Machine Learning Approaches](https://mlsanity.com/paper/2307.07695/) (2023)
- [Time series saliency maps: explaining models across multiple domains](https://mlsanity.com/paper/2505.13100/) (2025)
- [DeltaSHAP: Explaining Prediction Evolutions in Online Patient Monitoring with Shapley Values](https://mlsanity.com/paper/2507.02342/) (2025)
- [ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecasting](https://mlsanity.com/paper/2509.23159/) (2025)
- [Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models](https://mlsanity.com/paper/2603.10071/) (2026)
- [AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE](https://mlsanity.com/paper/2606.03631/) (2026)
- [Information Bottleneck Learning for Faithful Time Series Forecasting Explanations](https://mlsanity.com/paper/2607.28124/) (2026)
- [WinTSR: A Windowed Temporal Saliency Rescaling Method for Interpreting Time Series Deep Learning Models](https://mlsanity.com/paper/2412.04532/) (2024)
- [Evaluating Time Series Models with Knowledge Discovery](https://mlsanity.com/paper/2503.14869/) (2025)
- [PointExplainer: Towards Transparent Parkinson's Disease Diagnosis](https://mlsanity.com/paper/2505.03833/) (2025)
- [Robust Multi-Modal Forecasting: Integrating Static and Dynamic Features](https://mlsanity.com/paper/2505.15083/) (2025)
- [On the Necessity of Multi-Domain Explanation: An Uncertainty Principle Approach for Deep Time Series Models](https://mlsanity.com/paper/2506.03267/) (2025)
- [Delta-XAI: A Unified Framework for Explaining Prediction Changes in Online Time Series Monitoring](https://mlsanity.com/paper/2511.23036/) (2025)
- [A Self-explainable Model of Long Time Series by Extracting Informative Structured Causal Patterns](https://mlsanity.com/paper/2512.01412/) (2025)
- [When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate](https://mlsanity.com/paper/2512.03578/) (2025)

## Abstract (arXiv)

Interpreting time series models is uniquely challenging because it requires identifying both the location of time series signals that drive model predictions and their matching to an interpretable temporal pattern. While explainers from other modalities can be applied to time series, their inductive biases do not transfer well to the inherently challenging interpretation of time series. We present TimeX, a time series consistency model for training explainers. TimeX trains an interpretable surrogate to mimic the behavior of a pretrained time series model. It addresses the issue of model faithfulness by introducing model behavior consistency, a novel formulation that preserves relations in the latent space induced by the pretrained model with relations in the latent space induced by TimeX. TimeX provides discrete attribution maps and, unlike existing interpretability methods, it learns a latent space of explanations that can be used in various ways, such as to provide landmarks to visually aggregate similar explanations and easily recognize temporal patterns. We evaluate TimeX on eight synthetic and real-world datasets and compare its performance against state-of-the-art interpretability methods. We also conduct case studies using physiological time series. Quantitative evaluations demonstrate that TimeX achieves the highest or second-highest performance in every metric compared to baselines across all datasets. Through case studies, we show that the novel components of TimeX show potential for training faithful, interpretable models that capture the behavior of pretrained time series models.