# Dissecting Chronos: Sparse Autoencoders Reveal Causal Feature Hierarchies in Time Series Foundation Models

arXiv: https://arxiv.org/abs/2603.10071 · HTML: https://mlsanity.com/paper/2603.10071/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting time-series and weather models, one of 13 directions in Mechanistic interpretability.
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Placed in

- Mechanistic interpretability / [Interpreting time-series and weather models](https://mlsanity.com/mechinterp/interpreting-time-series-and-weather-models/)

## Builds on (most cited first, 19 of 19)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [Self-Interpretable Time Series Prediction with Counterfactual Explanations](https://mlsanity.com/paper/2306.06024/) (2023)
- [Learning Perturbations to Explain Time Series Predictions](https://mlsanity.com/paper/2305.18840/) (2023)
- [Explaining Time Series via Contrastive and Locally Sparse Perturbations](https://mlsanity.com/paper/2401.08552/) (2024)
- [Interpretation of Time-Series Deep Models: A Survey](https://mlsanity.com/paper/2305.14582/) (2023)
- [Interpretability for Time Series Transformers using A Concept Bottleneck Framework](https://mlsanity.com/paper/2410.06070/) (2024)
- [Towards Interpretable Time Series Foundation Models](https://mlsanity.com/paper/2507.07439/) (2025)
- [Mechanistic Interpretability for Transformer-based Time Series Classification](https://mlsanity.com/paper/2511.21514/) (2025)
- [Decoding Latent Spaces: Assessing the Interpretability of Time Series Foundation Models for Visual Analytics](https://mlsanity.com/paper/2504.20099/) (2025)
- [Time series saliency maps: explaining models across multiple domains](https://mlsanity.com/paper/2505.13100/) (2025)

## Built on by (most cited first, 2 of 2)

- [Superposition Is Not Necessary: A Mechanistic Interpretability Analysis of Transformer Representations for Time Series Forecasting](https://mlsanity.com/paper/2605.05151/) (2026)
- [Dynamical Parameters: An Interpretability Framework for Time-Series Foundation Models](https://mlsanity.com/paper/2609.34316/) (2026)

## Abstract (arXiv)

Time series foundation models (TSFMs) are increasingly deployed in high-stakes domains, yet their internal representations remain opaque. We present the first application of sparse autoencoders (SAEs) to a TSFM, training TopK SAEs on activations of Chronos-T5-Large (710M parameters) across six layers. Through 392 single-feature ablation experiments, we establish that every ablated feature produces a positive CRPS degradation, confirming causal relevance. Our analysis reveals a depth-dependent hierarchy: early encoder layers encode low-level frequency features, the mid-encoder concentrates causally critical change-detection features, and the final encoder compresses a rich but less causally important taxonomy of temporal concepts. The most critical features reside in the mid-encoder (max single-feature Delta CRPS = 38.61), not in the semantically richest final encoder layer, where progressive ablation paradoxically improves forecast quality. These findings demonstrate that mechanistic interpretability transfers effectively to TSFMs and that Chronos-T5 relies on abrupt-dynamics detection rather than periodic pattern recognition.