# Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model

arXiv: https://arxiv.org/abs/2506.11029 · HTML: https://mlsanity.com/paper/2506.11029/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Time series reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 62 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM reasoning / [Time series reasoning](https://mlsanity.com/lr/time-series-reasoning/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Better & Faster Large Language Models via Multi-token Prediction](https://mlsanity.com/paper/2404.19737/) (2024)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Do Large Language Models Latently Perform Multi-Hop Reasoning?](https://mlsanity.com/paper/2402.16837/) (2024)
- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding](https://mlsanity.com/paper/2410.01699/) (2024)
- [AutoTimes: Autoregressive Time Series Forecasters via Large Language Models](https://mlsanity.com/paper/2402.02370/) (2024)
- [From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting](https://mlsanity.com/paper/2501.02945/) (2025)
- [AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Data](https://mlsanity.com/paper/2310.20280/) (2023)

## Built on by (most cited first, 24 of 24)

- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [Moirai 2.0: When Less Is More for Time Series Forecasting](https://mlsanity.com/paper/2511.11698/) (2025)
- [Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting](https://mlsanity.com/paper/2602.17634/) (2026)
- [TiRex-2: Generalizing TiRex to Multivariate Data and Streaming](https://mlsanity.com/paper/2607.01204/) (2026)
- [Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models](https://mlsanity.com/paper/2602.06909/) (2026)
- [Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Models](https://mlsanity.com/paper/2509.25826/) (2025)
- [Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification](https://mlsanity.com/paper/2510.26777/) (2025)
- [Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attention](https://mlsanity.com/paper/2510.21795/) (2025)
- [EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Series](https://mlsanity.com/paper/2609.04239/) (2026)
- [Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?](https://mlsanity.com/paper/2510.16060/) (2025)
- [CITRAS-FM: Tiny Time Series Foundation Model for Covariate-Informed Zero-Shot Forecasting](https://mlsanity.com/paper/2606.10798/) (2026)
- [EIDOS: Latent-Space Predictive Learning for Time Series Foundation Models](https://mlsanity.com/paper/2602.14024/) (2026)
- [WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting](https://mlsanity.com/paper/2604.10544/) (2026)
- [Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding](https://mlsanity.com/paper/2605.20268/) (2026)
- [FinVerse: Financial Time-Series Benchmark](https://mlsanity.com/paper/2608.03259/) (2026)
- [Forecast Collapse in Time-Series Foundation Models](https://mlsanity.com/paper/2608.14106/) (2026)
- [TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicity](https://mlsanity.com/paper/2608.15767/) (2026)
- [TS-HINT: Enhancing Semiconductor Time Series Regression Using Attention Hints From Large Language Model Reasoning](https://mlsanity.com/paper/2512.05419/) (2025)
- [GlucoFM-Bench: Benchmarking Time-Series Foundation Models for Blood Glucose Forecasting](https://mlsanity.com/paper/2606.06881/) (2026)
- [Time Series as Language: A Universal Tokenizer for General-Purpose Time Series Foundation Models](https://mlsanity.com/paper/2606.09861/) (2026)
- [EXAONE Demand 1.0: A Time Series Foundation Model for Demand Forecasting](https://mlsanity.com/paper/2609.30880/) (2026)
- [Fracast-0: Fractal Weight Sharing for a Time Series Foundation Model with Only 85K Parameters](https://mlsanity.com/paper/2609.32209/) (2026)
- [A Unified Scaling Law for Time Series Foundation Models](https://mlsanity.com/paper/2610.05269/) (2026)
- [HART: Hybrid Attention-RWKV Time series Foundation Model](https://mlsanity.com/paper/2610.12883/) (2026)

## Abstract (arXiv)

We present a joint forecasting framework for time series prediction that contrasts with traditional direct or recursive methods. This framework achieves state-of-the-art performance for our designed foundation model, YingLong, and reveals a novel scaling effect: longer outputs significantly enhance model accuracy due to delayed chain-of-thought reasoning in our non-causal approach. YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery, aligning more effectively with language understanding tasks than with generation tasks. Additionally, we boost performance by tackling output variance with a multi-input ensemble. We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks on the ETT and Weather datasets. YingLong achieves more than 60% best performance. To ensure generalizability, we assessed the models using the GIFT-Eval benchmark, which comprises 23 time series datasets across 7 domains. Yinglong significantly outperformed the best time-series foundation models, end-to-end trained models by 14% and 44% in rank respectively.The pretrained 300M model is available at https://huggingface.co/qcw1314/YingLong_300m