mlsanityData to 7 Oct 2026

Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling

arXiv 2603.04791 ↗First posted 5 Mar 2026cs.AIcited by 24

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

We introduce Timer-S1, a strong Mixture-of-Experts (MoE) time series foundation model with 8.3B total parameters, 0.75B activated parameters for each token, and a context length of 11.5K. To overcome the scalability bottleneck in existing pre-trained time series foundation models, we perform Serial Scaling in three dimensions: model architecture, dataset, and training pipeline. Timer-S1 integrates sparse TimeMoE blocks and generic TimeSTP blocks for Serial-Token Prediction (STP), a generic training objective that adheres to the serial nature of forecasting. The proposed paradigm introduces serial computations to improve long-term predictions while avoiding costly rolling-style inference and pronounced error accumulation in the standard next-token prediction. Pursuing a high-quality and unbiased training dataset, we curate TimeBench, a corpus with one trillion time points, and apply meticulous data augmentation to mitigate predictive bias. We further pioneer a post-training stage, including continued pre-training and long-context extension, to enhance short-term and long-context performance. Evaluated on the large-scale GIFT-Eval leaderboard, Timer-S1 achieves state-of-the-art forecasting performance, attaining the best MASE and CRPS scores as a pre-trained model. Timer-S1 is released to facilitate further research.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 43 of its 61 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

43 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts14 Oct
  2. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts24 Sep

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  6. 2019Root Mean Square Layer Normalizationcited by 1,425
  7. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  8. 2020On Layer Normalization in the Transformer Architecturecited by 906
  9. 2024Chronos: Learning the Language of Time Seriescited by 712
  10. 2023A decoder-only foundation model for time-series forecastingcited by 650
  11. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  12. 2022Efficiently Scaling Transformer Inferencecited by 529
29 more
  1. 2022Transformers in Time Series: A Surveycited by 489
  2. 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
  3. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  4. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
  5. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  6. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  7. 2020Query-Key Normalization for Transformerscited by 266
  8. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  9. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  10. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
  11. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  12. 2025Chronos-2: From Univariate to Universal Forecastingcited by 158
  13. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
  14. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
  15. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  16. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 59
  17. 2024Toto: Time Series Optimized Transformer for Observabilitycited by 44
  18. 2024Scaling Law for Time Series Forecastingcited by 31
  19. 2024In-Context Fine-Tuning for Time-Series Foundation Modelscited by 31
  20. 2025ChronosX: Adapting Pretrained Time Series Models with Exogenous Variablescited by 21
  21. 2025VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoocited by 20
  22. 2025TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysiscited by 18
  23. 2025AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecastingcited by 15
  24. 2025The Serial Scaling Hypothesiscited by 11
  25. 2024PowerPM: Foundation Model for Power Systemscited by 9
  26. 2025FinCast: A Foundation Model for Financial Time-Series Forecastingcited by 9
  27. 2025TimeCopilotcited by 8
  28. 2025Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Modelscited by 7
  29. 2025CoRA: Covariate-Aware Adaptation of Time Series Foundation Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 24.

24 papers.

Sort:

By area:

  1. 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 10
  2. 2026Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modelingcited by 7
  3. 2026Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Modelscited by 5
  4. 2026Reasoning-Aware Training for Time Series Forecastingcited by 3
  5. 2026Into the ORBIT for Time Series: Training Regimes for Foundation Modelscited by 3
  6. 2026Don't Learn the Shape: Forecasting Periodic Time Series by Rank-1 Decompositioncited by 1
  7. 2026Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecastingcited by 1
  8. 2026TSQAgent: Rating Time Series Data Quality via Dedicated Agentic Reasoningcited by 1
  9. 2026Tabby: An Open Pretraining Recipe for Time Series Foundation Modelscited by 1
  10. 2026Tyan-WP: A Wind Power Foundation Model for Ultra-Short-Term Probabilistic Forecastingcited by 0
  11. 2026TimeVista: Exploring and Exploiting Vision-Language Models as Judges for Time Series Forecastingcited by 0
  12. 2026AgenticTwin: An Agentic LLM Framework Integrated with Digital Twin for Anomaly Detectioncited by 0
12 more
  1. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 0
  2. 2026Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Contextcited by 0
  3. 2026One Patch, Three Roles: What Is Actually Coupled in Autoregressive Time-Series Forecasting?cited by 0
  4. 2026t0: A Time-Series Foundation Model for Forecasting with Contextcited by 0
  5. 2026SGA: Uncertainty Quantification for Multi-Step Forecasting in Time Series Foundation Modelscited by 0
  6. 2026Aurora-X: Built for Extreme Time Series Forecastingcited by 0
  7. 2026Self-Evolving Time-Series Forecasting Agents with Episodic Memory and Online Policy Learningcited by 0
  8. 2026Instance-Adaptive Prompts as Context for Time-Series Foundation Modelscited by 0
  9. 2026Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Modelscited by 0
  10. 2026OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoningcited by 0
  11. 2026RideBench: A Large-Scale Exogenous-Aware Benchmark for Ride-Hailing Time Series Forecastingcited by 0
  12. 2026Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitivescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
  2. 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting
  3. 2025MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
  4. 2024TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting
  5. 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
  6. 2026TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale
  7. 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
  8. 2026Diversified Scaling Inference in Time Series Foundation Models