mlsanityData to 7 Oct 2026

Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models

arXiv 2602.06909 ↗First posted 6 Feb 2026cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

The recent surge in Time Series Foundation Models has rapidly advanced the field, yet the heterogeneous training setups across studies make it difficult to attribute improvements to architectural innovations versus data engineering. In this work, we investigate the potential of a standard patch Transformer, demonstrating that this generic architecture achieves state-of-the-art zero-shot forecasting performance using a straightforward training protocol. We conduct a comprehensive ablation study that covers model scaling, data composition, and training techniques to isolate the essential ingredients for high performance. Our findings identify the key drivers of performance, while confirming that the generic architecture itself demonstrates excellent scalability. By strictly controlling these variables, we provide comprehensive empirical results on model scaling across multiple dimensions. We release our open-source model and detailed findings to establish a transparent, reproducible baseline for future research.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  6. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  7. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  8. 2024Chronos: Learning the Language of Time Seriescited by 712
  9. 2023A decoder-only foundation model for time-series forecastingcited by 650
  10. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  11. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  12. 2024xLSTM: Extended Long Short-Term Memorycited by 320
15 more
  1. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
  2. 2021Monash Time Series Forecasting Archivecited by 256
  3. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
  4. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  5. 2025Chronos-2: From Univariate to Universal Forecastingcited by 158
  6. 2023LargeST: A Benchmark Dataset for Large-Scale Traffic Forecastingcited by 116
  7. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
  8. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  9. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 57
  10. 2023ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modelingcited by 46
  11. 2023BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecastingcited by 26
  12. 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Modelcited by 21
  13. 2023LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Predictioncited by 19
  14. 2023AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Datacited by 17
  15. 2024Abstracted Shapes as Tokens -- A Generalizable and Interpretable Model for Time-series Classificationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.