mlsanityData to 7 Oct 2026

Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model

arXiv 2506.11029 ↗First posted 20 May 2025cs.LG cs.AIcited by 21

On the map: LLM reasoning → Time series reasoning

Abstract (arXiv)

We present a joint forecasting framework for time series prediction that contrasts with traditional direct or recursive methods. This framework achieves state-of-the-art performance for our designed foundation model, YingLong, and reveals a novel scaling effect: longer outputs significantly enhance model accuracy due to delayed chain-of-thought reasoning in our non-causal approach. YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery, aligning more effectively with language understanding tasks than with generation tasks. Additionally, we boost performance by tackling output variance with a multi-input ensemble. We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks on the ETT and Weather datasets. YingLong achieves more than 60% best performance. To ensure generalizability, we assessed the models using the GIFT-Eval benchmark, which comprises 23 time series datasets across 7 domains. Yinglong significantly outperformed the best time-series foundation models, end-to-end trained models by 14% and 44% in rank respectively.The pretrained 300M model is available at https://huggingface.co/qcw1314/YingLong_300m

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Time series reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 62 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

21 of 62 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2020GLU Variants Improve Transformercited by 1,724
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2024Chronos: Learning the Language of Time Seriescited by 712
  6. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  7. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  8. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  9. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  10. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  11. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  12. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
9 more
  1. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  2. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  3. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
  4. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  5. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  6. 2024Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decodingcited by 65
  7. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 63
  8. 2025From Tables to Time: Extending TabPFN-v2 to Time Series Forecastingcited by 49
  9. 2023AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Datacited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 14.

21 papers.

Sort:

By area:

  1. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
  2. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 59
  3. 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 11
  4. 2026Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Modelscited by 10
  5. 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 10
  6. 2025Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classificationcited by 9
  7. 2025Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Modelscited by 8
  8. 2025Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attentioncited by 7
  9. 2025Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?cited by 2
  10. 2026EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Seriescited by 2
  11. 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecastingcited by 1
  12. 2026CITRAS-FM: Tiny Time Series Foundation Model for Covariate-Informed Zero-Shot Forecastingcited by 1
9 more
  1. 2026FinVerse: Financial Time-Series Benchmarkcited by 1
  2. 2026Forecast Collapse in Time-Series Foundation Modelscited by 1
  3. 2026TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicitycited by 1
  4. 2025TS-HINT: Enhancing Semiconductor Time Series Regression Using Attention Hints From Large Language Model Reasoningcited by 0
  5. 2026EIDOS: Latent-Space Predictive Learning for Time Series Foundation Modelscited by 0
  6. 2026GlucoFM-Bench: Benchmarking Time-Series Foundation Models for Blood Glucose Forecastingcited by 0
  7. 2026Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Modelscited by 0
  8. 2026EXAONE Demand 1.0: A Time Series Foundation Model for Demand Forecastingcited by 0
  9. 2026Fracast-0: Fractal Weight Sharing for a Time Series Foundation Model with Only 85K Parameterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
  2. 2026Latent Chain-of-Thought Improves Structured-Data Transformers
  3. 2025TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
  4. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models
  5. 2026Thoth: Mid-Training Bridges LLMs to Time Series Understanding
  6. 2025Can Slow-thinking LLMs Reason Over Time? Empirical Studies in Time Series Forecasting
  7. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts
  8. 2024Implicit Reasoning in Deep Time Series Forecasting