# Rethinking Multimodal Time-Series Forecasting Evaluation

arXiv: https://arxiv.org/abs/2607.06973 · HTML: https://mlsanity.com/paper/2607.06973/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [OpenThoughts: Data Recipes for Reasoning Models](https://mlsanity.com/paper/2506.04178/) (2025)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [LSTPrompt: Large Language Models as Zero-Shot Time Series Forecasters by Long-Short-Term Prompting](https://mlsanity.com/paper/2402.16132/) (2024)
- [MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering](https://mlsanity.com/paper/2503.16858/) (2025)
- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insights](https://mlsanity.com/paper/2503.01895/) (2025)
- [FutureX-Pro: Extending Future Prediction to High-Value Vertical Domains](https://mlsanity.com/paper/2601.12259/) (2026)
- [MoTime: A Dataset Suite for Multimodal Time Series Forecasting](https://mlsanity.com/paper/2505.15072/) (2025)
- [ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset](https://mlsanity.com/paper/2505.10083/) (2025)

## Built on by (most cited first, 7 of 7)

- [Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context](https://mlsanity.com/paper/2609.15087/) (2026)
- [LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models](https://mlsanity.com/paper/2607.24892/) (2026)
- [TimeLitmus: A Diagnostic Benchmark for Cross-Modal Understanding and Explanation Faithfulness in Event-Conditioned Time-Series Prediction](https://mlsanity.com/paper/2609.24677/) (2026)
- [KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting](https://mlsanity.com/paper/2610.12738/) (2026)
- [World2TS: Real-World Event Context for Time-Series Forecasting](https://mlsanity.com/paper/2610.14861/) (2026)
- [MMTA: Benchmarking Multimodal Temporal Analysis with Time Series, Text, and Vision](https://mlsanity.com/paper/2610.17856/) (2026)
- [TimeRSI: Towards Self-Improved Agentic Deployment for Time-Series Foundation Models](https://mlsanity.com/paper/2610.19725/) (2026)

## Abstract (arXiv)

We introduce a new context-enriched, multimodal time series forecasting benchmark, TimesX. TimesX contains a wide selection of high-quality real-world time series with diverse domains and textual contexts obtained from an automated data generation pipeline, which helps address three main issues of existing multimodal forecasting benchmarks: (1) poor generalization due to the small scale and synthetic nature of benchmark data, (2) very limited types of textual contexts in the benchmarks, and (3) an inability to mitigate data leakage in evaluation. We conduct a thorough empirical study of zero-shot multimodal forecasting approaches on TimesX. Our results suggest that many approaches that perform well on existing benchmarks may fail on TimesX. In contrast, simple ensemble methods that leverage rich textual context accompanying time-series can outperform strong baselines on TimesX.