mlsanityData to 14 Oct 2026

TSAE: Structured Sparse Autoencoders for Interpreting Time-Series Forecasting Models

arXiv 2610.04925 ↗First posted 4 Oct 2026cs.LG cs.AIcited by 0

On the map: Mechanistic interpretability → Interpreting time-series and weather models

Abstract (arXiv)

Time-series forecasting informs critical decisions in energy dispatch, industrial operations, and environmental monitoring; understanding the patterns models rely on is essential for assessing reliability and identifying failures. Input attribution identifies important variables and time segments but offers limited insight into internal features, while standard sparse autoencoder (SAE) objectives do not directly constrain cross-variable structure or temporal continuity. We introduce TSAE, a structured sparse autoencoder for forecasting representations that decomposes hidden states into individually inspectable features. TSAE organizes cross-variable structure through shared and variable-routed private dictionaries, separates feature detection from magnitude estimation with gated encoding, and constrains neighboring sparse-code changes according to raw-segment similarity. These mechanisms support analysis of variable context, activation strength, and temporal evolution. Forecast-consistency fine-tuning further improves preservation of the frozen forecaster's outputs. The accompanying TSEVAL protocol separately audits fidelity, feature coherence, and physical calibration to ground feature interpretation. In three-seed experiments with frozen PatchTST on ETTh1, ETTh2, and ETTm1, TSAE achieves the lowest hidden-state reconstruction error, normalized forecast-reconstruction error (NFRE), and feature transition rate among five SAEs at comparable per-token activity. NFRE decreases by 4.3-26.4% relative to the next-best mean. Dataset-dependent tradeoffs in selectivity, physical correlation, and calibration show that fidelity and temporal-stability gains require independent semantic validation to support feature interpretation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpreting time-series and weather models, one of 13 directions in Mechanistic interpretability. Builds on 14 of its 17 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

14 of 17 references.

References · 14

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  4. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  5. 2018Sanity Checks for Saliency Mapscited by 1,156
  6. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  7. 2024Scaling and evaluating sparse autoencoderscited by 553
  8. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  9. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  10. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  11. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 179
  12. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
2 more
  1. 2020TimeSHAP: Explaining Recurrent Models through Sequence Perturbationscited by 74
  2. 2021Explaining Time Series Predictions with Dynamic Maskscited by 73

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models
  2. 2026Superposition Is Not Necessary: A Mechanistic Interpretability Analysis of Transformer Representations for Time Series Forecasting
  3. 2026TimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Models
  4. 2026AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
  5. 2026PatchFormer: A Patch-Based Time Series Foundation Model with Hierarchical Masked Reconstruction and Cross-Domain Transfer Learning for Zero-Shot Multi-Horizon Forecasting
  6. 2026MoHETS: Long-term Time Series Forecasting with Mixture-of-Heterogeneous-Experts
  7. 2026Aurora-X: Built for Extreme Time Series Forecasting
  8. 2026Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE