mlsanityData to 14 Oct 2026

Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges

arXiv 2510.13654 ↗First posted 15 Oct 2025cs.LG cs.AIcited by 16

On the map: LLM reasoning → Time series reasoning

Abstract (arXiv)

Time Series Foundation Models (TSFMs) represent a new paradigm for time-series forecasting, promising zero-shot predictions without the need for task-specific training or fine-tuning. However, similar to Large Language Models (LLMs), the evaluation of TSFMs is challenging: as training corpora grow increasingly large, it becomes difficult to ensure the integrity of the test sets used for benchmarking. An investigation of existing TSFM evaluation studies identifies two kinds of information leakage: (1) train-test sample overlaps arising from the multi-purpose reuse of datasets and (2) temporal overlap of correlated train and test series. Ignoring these forms of information leakage when benchmarking TSFMs risks producing overly optimistic performance estimates that fail to generalize to real-world settings. We therefore argue for the development of novel evaluation methodologies that avoid pitfalls already observed in both LLM and classical time-series benchmarking, and we call on the research community to adopt principled approaches to safeguard the integrity of TSFM evaluation.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Time series reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 52 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

42 of 52 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  5. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  6. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  7. 2024Chronos: Learning the Language of Time Seriescited by 759
  8. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  9. 2023A decoder-only foundation model for time-series forecastingcited by 692
  10. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  11. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  12. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
30 more
  1. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  2. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  3. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  4. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  5. 2021Monash Time Series Forecasting Archivecited by 267
  6. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 249
  7. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  8. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  9. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  10. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  11. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  12. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  13. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  14. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
  15. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  16. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 88
  17. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
  18. 2025fev-bench: A Realistic Benchmark for Time Series Forecastingcited by 64
  19. 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
  20. 2020Kaggle forecasting competitions: An overlooked learning opportunitycited by 56
  21. 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
  22. 2022Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practicescited by 55
  23. 2023An Open Source Data Contamination Report for Large Language Modelscited by 51
  24. 2024In-Context Fine-Tuning for Time-Series Foundation Modelscited by 35
  25. 2024Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfercited by 32
  26. 2025LightGTS: A Lightweight General Time Series Forecasting Modelcited by 29
  27. 2024Scaling-laws for Large Time-series Modelscited by 18
  28. 2024Cherry-Picking in Time Series Forecasting: How to Select Datasets to Make Your Model Shinecited by 10
  29. 2025Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecastingcited by 10
  30. 2020Enhanced spatio-temporal electric load forecasts using less data with active deep learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 14.

16 papers.

Sort:

By area:

  1. 2026QuitoBench: A High-Quality Open Time Series Forecasting Benchmarkcited by 10
  2. 2025Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoningcited by 4
  3. 2026Position: Universal Time Series Foundation Models Rest on a Category Errorcited by 4
  4. 2025TS-Arena -- A Live Forecast Pre-Registration Platformcited by 3
  5. 2026TSFMAudit: Data Contamination Auditing in Forecasting Time Series Foundation Modelscited by 3
  6. 2026Aionoscope: Debugging Latent-State Accessibility in Time-Series Representationscited by 2
  7. 2026Evaluating Time Series Foundation Models for Electricity Price Forecasting: Contamination Risk, Distributional Shifts, and Covariate Dependencecited by 1
  8. 2026LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applicationscited by 1
  9. 2026MACROCAST: A Vintage-Consistent Time Series Foundation Model for Real-Time Macroeconomic Forecastingcited by 0
  10. 2026The Illusion of Replacement: Rethinking Specialized Machine Learning Models in the Foundation Model Eracited by 0
  11. 2026A Later Test Set Is Not a New Domain: Pretraining Familiarity Survives a Contamination-Free Hold-Outcited by 0
  12. 2026Tracing the Evidence Behind Zero-Shot Time-Series Forecasting: A Source-First Taxonomy and Audit Frameworkcited by 0
4 more
  1. 2026A Unified Scaling Law for Time Series Foundation Modelscited by 0
  2. 2026The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptationcited by 0
  3. 2026Are We Really Benchmarking Forecasting Models? The Impact of Preprocessing on Time Series Performancecited by 0
  4. 2026On the Encoding of Calendar Features in Covariate-Informed Time Series Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Conversational Time Series Foundation Models: Towards Explainable and Effective Forecasting
  2. 2026When, Not How Much: Evaluating Time-Series Foundation Models on Sparse Events
  3. 2026Rethinking the Role of LLMs in Time Series Forecasting
  4. 2025Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models
  5. 2026Universal Redundancies in Time Series Foundation Models
  6. 2026Foundations without Fundamentals: Zero-Shot Blind Spots in Time Series FMs
  7. 2026GRAM: Correcting Frozen Time-Series Foundation Models via Graph-Retrieved Amplitude Memory
  8. 2025The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models