# Non-Stationarity in the Embedding Space of Time Series Foundation Models

arXiv: https://arxiv.org/abs/2604.16428 · HTML: https://mlsanity.com/paper/2604.16428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 11 of 11)

- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis](https://mlsanity.com/paper/2410.16032/) (2024)
- [Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting](https://mlsanity.com/paper/2302.14829/) (2023)
- [TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting](https://mlsanity.com/paper/2410.04442/) (2024)
- [TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis](https://mlsanity.com/paper/2402.16412/) (2024)
- [Deep Adaptive Input Normalization for Time Series Forecasting](https://mlsanity.com/paper/1902.07892/) (2019)

## Abstract (arXiv)

Time series foundation models (TSFMs) are widely used as generic feature extractors, yet the notion of non-stationarity in their embedding spaces remains poorly understood. Recent work often conflates non-stationarity with distribution shift, blurring distinctions fundamental to classical time-series analysis and long-standing methodologies such as statistical process control (SPC). In SPC, non-stationarity signals a process leaving a stable regime - via shifts in mean, variance, or emerging trends - and detecting such departures is central to quality monitoring and change-point analysis. Motivated by this diagnostic tradition, we study how different forms of distributional non-stationarity - mean shifts, variance changes, and linear trends - become linearly accessible in TSFM embedding spaces under controlled conditions. We further examine temporal non-stationarity arising from persistence, which reflects violations of weak stationarity due to long-memory or near-unit-root behavior rather than explicit distributional shifts. By sweeping shift strength and probing multiple TSFMs, we find that embedding-space detectability of non-stationarity degrades smoothly and that different models exhibit distinct, model-specific failure modes.