Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models
Not on the map. Why?
Abstract (arXiv)
The recent surge in Time Series Foundation Models has rapidly advanced the field, yet the heterogeneous training setups across studies make it difficult to attribute improvements to architectural innovations versus data engineering. In this work, we investigate the potential of a standard patch Transformer, demonstrating that this generic architecture achieves state-of-the-art zero-shot forecasting performance using a straightforward training protocol. We conduct a comprehensive ablation study that covers model scaling, data composition, and training techniques to isolate the essential ingredients for high performance. Our findings identify the key drivers of performance, while confirming that the generic architecture itself demonstrates excellent scalability. By strictly controlling these variables, we provide comprehensive empirical results on model scaling across multiple dimensions. We release our open-source model and detailed findings to establish a transparent, reproducible baseline for future research.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 6 Feb 2026.
Builds on
27 of 37 references.
References · 27
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
- 2022Simplified State Space Layers for Sequence Modelingcited by 776
- 2024Chronos: Learning the Language of Time Seriescited by 712
- 2023A decoder-only foundation model for time-series forecastingcited by 650
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2024xLSTM: Extended Long Short-Term Memorycited by 320
15 more
- 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
- 2021Monash Time Series Forecasting Archivecited by 256
- 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
- 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
- 2025Chronos-2: From Univariate to Universal Forecastingcited by 158
- 2023LargeST: A Benchmark Dataset for Large-Scale Traffic Forecastingcited by 116
- 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
- 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
- 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 57
- 2023ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modelingcited by 46
- 2023BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecastingcited by 26
- 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Modelcited by 21
- 2023LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Predictioncited by 19
- 2023AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Datacited by 17
- 2024Abstracted Shapes as Tokens -- A Generalizable and Interpretable Model for Time-series Classificationcited by 12
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 11
- 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 10
- 2026TimeRouter: Efficient and Adaptive Routing of Time-Series Foundation Modelscited by 2
- 2026Tabby: An Open Pretraining Recipe for Time Series Foundation Modelscited by 1
- 2026Unified Zero-Shot Time Series Forecasting: A Darts Foundationcited by 0
- 2026FlowTSFM: Turning Encoder Depth into Quantile Transportcited by 0
- 2026Instance-Adaptive Prompts as Context for Time-Series Foundation Modelscited by 0
- 2026QiYao-M: Multimodal Time Series Foundation Model with Role-Aware Modeling of Endogenous and Exogenous Modalitiescited by 0
- 2026OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoningcited by 0
- 2026Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitivescited by 0
- 2026Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives8 Oct 2026
- 2026OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning30 Sep 2026
- 2026QiYao-M: Multimodal Time Series Foundation Model with Role-Aware Modeling of Endogenous and Exogenous Modalities28 Sep 2026
- 2026Instance-Adaptive Prompts as Context for Time-Series Foundation Models28 Sep 2026
- 2026Tabby: An Open Pretraining Recipe for Time Series Foundation Models12 Sep 2026
- 2026FlowTSFM: Turning Encoder Depth into Quantile Transport12 Sep 2026
- 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streaming1 Jul 2026
- 2026Unified Zero-Shot Time Series Forecasting: A Darts Foundation25 Jun 2026
- 2026TimeRouter: Efficient and Adaptive Routing of Time-Series Foundation Models10 Jun 2026
- 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting19 Feb 2026