# Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecasting

arXiv: https://arxiv.org/abs/2509.24789 · HTML: https://mlsanity.com/paper/2509.24789/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [Sundial: A Family of Highly Capable Time Series Foundation Models](https://mlsanity.com/paper/2502.00816/) (2025)
- [Qwen2.5-1M Technical Report](https://mlsanity.com/paper/2501.15383/) (2025)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://mlsanity.com/paper/2409.10173/) (2024)
- [ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data](https://mlsanity.com/paper/2412.11376/) (2024)
- [From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection](https://mlsanity.com/paper/2409.17515/) (2024)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis](https://mlsanity.com/paper/2406.08627/) (2024)
- [TSFM-Bench: A Comprehensive and Unified Benchmark of Foundation Models for Time Series Forecasting](https://mlsanity.com/paper/2410.11802/) (2024)

## Built on by (most cited first, 10 of 10)

- [Toto 2.0: Time Series Forecasting Enters the Scaling Era](https://mlsanity.com/paper/2605.20119/) (2026)
- [Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges](https://mlsanity.com/paper/2510.13654/) (2025)
- [Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models](https://mlsanity.com/paper/2606.14222/) (2026)
- [Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning](https://mlsanity.com/paper/2511.02748/) (2025)
- [Position: Universal Time Series Foundation Models Rest on a Category Error](https://mlsanity.com/paper/2602.05287/) (2026)
- [Navigating the Safety-Fidelity Trade-off: Massive-Variate Time Series Forecasting for Power Systems via Probabilistic Scenarios](https://mlsanity.com/paper/2606.13338/) (2026)
- [TRACE: A Reproducible Benchmark for Electricity Price Forecasting with Official Operational Text](https://mlsanity.com/paper/2610.02256/) (2026)
- [Pythia: Toward Foundation World Models for Multimodal Time Series](https://mlsanity.com/paper/2610.05240/) (2026)
- [A Unified Scaling Law for Time Series Foundation Models](https://mlsanity.com/paper/2610.05269/) (2026)
- [GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation](https://mlsanity.com/paper/2610.17794/) (2026)

## Abstract (arXiv)

The evaluation of time series forecasting models is hindered by a lack of high-quality benchmarks, leading to overestimated assessments of progress. Existing datasets suffer from issues ranging from small-scale, low-frequency, pre-training data contamination in unimodal designs to the temporal and description leakage prevalent in early multimodal designs. To address this, we formalize the core principles of high-fidelity benchmarking, focusing on data sourcing integrity, leak-free design, and structural clarity. We introduce Fidel-TS, a new large-scale benchmark built from these principles. Our experiments reveal the limitations of prior benchmarks and the potential discrepancies in model evaluation, providing new insights into multiple existing unimodal and multimodal forecasting models and LLMs across various evaluation tasks.