Diversified Scaling Inference in Time Series Foundation Models
On the map: LLM reasoning → Diffusion reasoning and test-time scaling
Abstract (arXiv)
The advancement of Time Series Foundation Models (TSFMs) has been driven primarily by large-scale pre-training, but inference-time compute potential remains largely untapped. This work systematically investigates two questions: how do TSFMs behave under standard sampling-based inference scaling, and can controlled sampling diversity enhance performance? We first examine the properties of TSFMs under standard sampling often fail to adhere to scaling laws due to insufficient exploration of the solution space. Building on this, we then delve into diversified inference scaling via tailored time series perturbations to expand the generative distribution's support. We theoretically analyze the diversity-fidelity trade-off and derive a critical sample threshold for diversified sampling to outperform standard sampling. Extensive experiments across various TSFMs and datasets show proper diversified inference scaling yields substantial performance gains without parameter updates, establishing inference design as a critical, compute-efficient dimension of TSFM optimization. As an application, we propose RobustMSE, a rigorous metric to quantify the headroom performance of TSFM under a fixed budget. Overall, our findings clarify these factor interactions, enabling reliable performance via diverse large-scale inference time series in parallel environments without re-training TSFMs.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 40 of its 47 references that are in the map. First posted on arXiv 24 Jan 2026.
Builds on
40 of 47 references.
References · 40
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
28 more
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
- 2024Chronos: Learning the Language of Time Seriescited by 712
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2023A decoder-only foundation model for time-series forecastingcited by 650
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2021True Few-Shot Learning with Language Modelscited by 343
- 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
- 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
- 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
- 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 241
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
- 2023TimeGPT-1cited by 149
- 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
- 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
- 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 55
- 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 30
- 2024Protecting Your LLMs with Information Bottleneckcited by 26
- 2024Explaining Time Series via Contrastive and Locally Sparse Perturbationscited by 22
- 2025Foundation Models for Time Series: A Surveycited by 19
- 2024RobustTSF: Towards Theory and Design of Robust Time Series Forecasting with Anomaliescited by 17
- 2025Intent Factored Generation: Unleashing the Diversity in Your Language Modelcited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
- 2025On the Effect of Sampling Diversity in Scaling LLM Inference
- 2026GRAM: Correcting Frozen Time-Series Foundation Models via Graph-Retrieved Amplitude Memory
- 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithms
- 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model
- 2026Diversify Diffusion with Temperature Sampling and Variance-Corrective Time Shifting
- 2026Inference-Time Scaling for Visual AutoRegressive modeling by Searching Representative Samples
- 2025Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges