# On the Internal Semantics of Time-Series Foundation Models

arXiv: https://arxiv.org/abs/2511.15324 · HTML: https://mlsanity.com/paper/2511.15324/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 10 of 10)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [Exploring Representations and Interventions in Time Series Foundation Models](https://mlsanity.com/paper/2409.12915/) (2024)

## Built on by (most cited first, 5 of 5)

- [Explainable Load Forecasting with Covariate-Informed Time Series Foundation Models](https://mlsanity.com/paper/2604.28149/) (2026)
- [Interpretability in Deep Time Series Models Demands Semantic Alignment](https://mlsanity.com/paper/2602.02239/) (2026)
- [Dynamical Parameters: An Interpretability Framework for Time-Series Foundation Models](https://mlsanity.com/paper/2609.34316/) (2026)
- [Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Models](https://mlsanity.com/paper/2609.39445/) (2026)
- [What Does an Observability Foundation Model Know?](https://mlsanity.com/paper/2610.05577/) (2026)

## Abstract (arXiv)

Time-series Foundation Models (TSFMs) have recently emerged as a universal paradigm for learning across diverse temporal domains. However, despite their empirical success, the internal mechanisms by which these models represent fundamental time-series concepts remain poorly understood. In this work, we undertake a systematic investigation of concept interpretability in TSFMs. Specifically, we examine: (i) which layers encode which concepts, (ii) whether concept parameters are linearly recoverable, (iii) how representations evolve in terms of concept disentanglement and abstraction across model depth, and (iv) how models process compositions of concepts. We systematically probe these questions using layer-wise analyses, linear recoverability tests, and representation similarity measures, providing a structured account of TSFM semantics. The resulting insights show that early layers mainly capture local, time-domain patterns (e.g., AR(1), level shifts, trends), while deeper layers encode dispersion and change-time signals, with spectral and warping factors remaining the hardest to recover linearly. In compositional settings, however, probe performance degrades, revealing interference between concepts. This highlights that while atomic concepts are reliably localized, composition remains a challenge, underscoring a key limitation in current TSFMs' ability to represent interacting temporal phenomena.