mlsanityData to 14 Oct 2026

A Unified Scaling Law for Time Series Foundation Models

arXiv 2610.05269 ↗First posted 4 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We develop a Unified Scaling Law and a Unified Theory of Time Series Learning to understand how model capacity and historical information support forecasting. Across different lookback lengths and forecast horizons, we analyze 18,768 experimental cells from 21 checkpoints on 23 dataset-frequency tasks spanning six domains. Our empirical methodology integrates local resource relations into a parsimonious, fitted five-parameter law: capacity gains increase with history, context gains diminish toward saturation, and horizon effects enter as a common shift. Fitted without Toto 2.0, the law predicts its horizon-averaged capacity-scaling curves with mean absolute percentage errors of 1.09% and 1.50% at input lengths 2048 and 4096. To understand how history supports prediction, our learning theory uses Gaussian regression to analyze rule identification and predictive capability. We hypothesize that full-shot models learn by accumulating information in weights, while frozen time series foundation models (TSFMs) use history by extracting information through activations. Matched-history comparisons establish the predictive value of additional history. Controlled parameter exchanges and activation interventions provide evidence that history-derived rule information can be retained, reused across queries, and used to recover a contribution to long-context prediction. Together, these findings inform capacity scaling, context allocation, and the development of models that retain and apply historical rules. Code and main results are available at https://github.com/Fifthky/UniScale.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

55 of 67 references.

References · 55

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  3. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  4. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  5. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  6. 2024Chronos: Learning the Language of Time Seriescited by 759
  7. 2023A decoder-only foundation model for time-series forecastingcited by 692
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  9. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  10. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  11. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  12. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
38 more
  1. 2025Chronos-2: From Univariate to Universal Forecastingcited by 194
  2. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  3. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  4. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  5. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 87
  6. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 69
  7. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
  8. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  9. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 60
  10. 2024In-Context Fine-Tuning for Time-Series Foundation Modelscited by 35
  11. 2024Scaling Law for Time Series Forecastingcited by 33
  12. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 33
  13. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  14. 2024Zero-shot forecasting of chaotic systemscited by 28
  15. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 28
  16. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 24
  17. 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Modelcited by 24
  18. 2025Foundation Models for Time Series: A Surveycited by 23
  19. 2024Scaling-laws for Large Time-series Modelscited by 18
  20. 2025Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learningcited by 18
  21. 2024How do Transformers perform In-Context Autoregressive Learning?cited by 17
  22. 2025Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challengescited by 16
  23. 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 13
  24. 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 13
  25. 2026Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Modelscited by 12
  26. 2025From Samples to Scenarios: A New Paradigm for Probabilistic Forecastingcited by 10
  27. 2025Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecastingcited by 10
  28. 2026QuitoBench: A High-Quality Open Time Series Forecasting Benchmarkcited by 10
  29. 2026Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modelingcited by 10
  30. 2025Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Modelscited by 9
  31. 2025Understanding Transformers for Time Series: Rank Structure, Flow-of-ranks, and Compressibilitycited by 9
  32. 2026Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Modelscited by 9
  33. 2025FlowState: Sampling-Rate-Equivariant Time-Series Forecastingcited by 8
  34. 2025Re(Visiting) Time Series Foundation Models in Financecited by 7
  35. 2025Exploring Accuracy Law for Deep Time Series Forecasters: An Empirical Studycited by 5
  36. 2026Position: Universal Time Series Foundation Models Rest on a Category Errorcited by 4
  37. 2026Diversified Scaling Inference in Time Series Foundation Modelscited by 3
  38. 2026Why Do Time Series Models Need Long Context Windows?cited by 3

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.