mlsanityData to 14 Oct 2026

Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting

arXiv 2509.24789 ↗First posted 29 Sep 2025cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

The evaluation of time series forecasting models is hindered by a lack of high-quality benchmarks, leading to overestimated assessments of progress. Existing datasets suffer from issues ranging from small-scale, low-frequency, pre-training data contamination in unimodal designs to the temporal and description leakage prevalent in early multimodal designs. To address this, we formalize the core principles of high-fidelity benchmarking, focusing on data sourcing integrity, leak-free design, and structural clarity. We introduce Fidel-TS, a new large-scale benchmark built from these principles. Our experiments reveal the limitations of prior benchmarks and the potential discrepancies in model evaluation, providing new insights into multiple existing unimodal and multimodal forecasting models and LLMs across various evaluation tasks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

32 of 50 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  11. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  12. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
20 more
  1. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  2. 2024Chronos: Learning the Language of Time Seriescited by 759
  3. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  4. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  5. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  6. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  7. 2022Data Contamination: From Memorization to Exploitationcited by 236
  8. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 197
  9. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  10. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  11. 2025Qwen2.5-1M Technical Reportcited by 164
  12. 2023TimeGPT-1cited by 158
  13. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
  14. 2024ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Datacited by 100
  15. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
  16. 2024Context is Key: A Benchmark for Forecasting with Essential Textual Informationcited by 87
  17. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 74
  18. 2024TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecastingcited by 58
  19. 2024LLM Processes: Numerical Predictive Distributions Conditioned on Natural Languagecited by 55
  20. 2024Intervention-Aware Forecasting: Breaking Historical Limits from a System Perspectivecited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2026Toto 2.0: Time Series Forecasting Enters the Scaling Eracited by 30
  2. 2025Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challengescited by 16
  3. 2026Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Modelscited by 9
  4. 2025Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoningcited by 4
  5. 2026Position: Universal Time Series Foundation Models Rest on a Category Errorcited by 4
  6. 2026Navigating the Safety-Fidelity Trade-off: Massive-Variate Time Series Forecasting for Power Systems via Probabilistic Scenarioscited by 0
  7. 2026TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Textcited by 0
  8. 2026Pythia: Toward Foundation World Models for Multimodal Time Seriescited by 0
  9. 2026A Unified Scaling Law for Time Series Foundation Modelscited by 0
  10. 2026GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluationcited by 0