# MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering

arXiv: https://arxiv.org/abs/2503.16858 · HTML: https://mlsanity.com/paper/2503.16858/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in Time series reasoning, one of 17 directions in LLM reasoning.
- Builds on 37 of its 59 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Placed in

- LLM reasoning / [Time series reasoning](https://mlsanity.com/lr/time-series-reasoning/)

## Builds on (most cited first, 30 of 37)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts](https://mlsanity.com/paper/2409.16040/) (2024)
- [WeatherBench: A benchmark dataset for data-driven weather forecasting](https://mlsanity.com/paper/2002.00469/) (2020)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [WeatherBench 2: A benchmark for the next generation of data-driven global weather models](https://mlsanity.com/paper/2308.15560/) (2023)
- [PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance](https://mlsanity.com/paper/2306.05443/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [Are Language Models Actually Useful for Time Series Forecasting?](https://mlsanity.com/paper/2406.16964/) (2024)
- [FinBen: A Holistic Financial Benchmark for Large Language Models](https://mlsanity.com/paper/2402.12659/) (2024)
- [LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters](https://mlsanity.com/paper/2308.08469/) (2023)
- [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://mlsanity.com/paper/2409.19839/) (2024)
- [ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data](https://mlsanity.com/paper/2412.11376/) (2024)
- [From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection](https://mlsanity.com/paper/2409.17515/) (2024)
- [Timer-XL: Long-Context Transformers for Unified Time Series Forecasting](https://mlsanity.com/paper/2410.04803/) (2024)
- [Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement](https://mlsanity.com/paper/2503.01875/) (2025)
- [Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting](https://mlsanity.com/paper/2306.11025/) (2023)
- [Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis](https://mlsanity.com/paper/2406.08627/) (2024)
- [TimeSeriesExam: A time series understanding exam](https://mlsanity.com/paper/2410.14752/) (2024)
- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark](https://mlsanity.com/paper/2404.16563/) (2024)
- [TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents](https://mlsanity.com/paper/2502.11418/) (2025)
- [Large language models can be zero-shot anomaly detectors for time series?](https://mlsanity.com/paper/2405.14755/) (2024)
- [Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations](https://mlsanity.com/paper/2302.10493/) (2023)

## Built on by (most cited first, 30 of 55)

- [TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models](https://mlsanity.com/paper/2509.24803/) (2025)
- [TimeART: Towards Agentic Time Series Reasoning via Tool-Augmentation](https://mlsanity.com/paper/2601.13653/) (2026)
- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series](https://mlsanity.com/paper/2506.10412/) (2025)
- [TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Models](https://mlsanity.com/paper/2601.18744/) (2026)
- [SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding](https://mlsanity.com/paper/2603.18567/) (2026)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference](https://mlsanity.com/paper/2509.01822/) (2025)
- [BEDTime: A Unified Benchmark for Automatically Describing Time Series](https://mlsanity.com/paper/2509.05215/) (2025)
- [TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning](https://mlsanity.com/paper/2510.03519/) (2025)
- [When Does Multimodality Lead to Better Time Series Forecasting?](https://mlsanity.com/paper/2506.21611/) (2025)
- [HEARTS: Benchmarking LLM Reasoning on Health Time Series](https://mlsanity.com/paper/2603.06638/) (2026)
- [TS-Agent: Understanding and Reasoning Over Raw Time Series via Iterative Insight Gathering](https://mlsanity.com/paper/2510.07432/) (2025)
- [Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision](https://mlsanity.com/paper/2506.00807/) (2025)
- [STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learning](https://mlsanity.com/paper/2601.03248/) (2026)
- [SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training](https://mlsanity.com/paper/2605.08738/) (2026)
- [Rethinking Multimodal Time-Series Forecasting Evaluation](https://mlsanity.com/paper/2607.06973/) (2026)
- [TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems](https://mlsanity.com/paper/2604.05364/) (2026)
- [TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale](https://mlsanity.com/paper/2604.10291/) (2026)
- [TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis](https://mlsanity.com/paper/2510.06063/) (2025)
- [Overcoming the Modality Gap in Context-Aided Forecasting](https://mlsanity.com/paper/2603.12451/) (2026)
- [AXIS: Explainable Time Series Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2509.24378/) (2025)
- [Thoth: Mid-Training Bridges LLMs to Time Series Understanding](https://mlsanity.com/paper/2603.01042/) (2026)
- [FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models](https://mlsanity.com/paper/2605.03460/) (2026)
- [CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Series](https://mlsanity.com/paper/2607.09880/) (2026)
- [Time Series Reasoning via Process-Verifiable Thinking Data Synthesis and Scheduling for Tailored LLM Reasoning](https://mlsanity.com/paper/2602.07830/) (2026)
- [LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics](https://mlsanity.com/paper/2604.17295/) (2026)
- [Towards Interpretable and Trustworthy Time Series Reasoning: A BlueSky Vision](https://mlsanity.com/paper/2510.16980/) (2025)
- [SenTSR-Bench: Thinking with Injected Knowledge for Time-Series Reasoning](https://mlsanity.com/paper/2602.19455/) (2026)
- [TimeClaw: A Time-Series AI Agent with Exploratory Execution Learning](https://mlsanity.com/paper/2605.10038/) (2026)

## Abstract (arXiv)

Understanding the relationship between textual news and time-series evolution is a critical yet under-explored challenge in applied data science. While multimodal learning has gained traction, existing multimodal time-series datasets fall short in evaluating cross-modal reasoning and complex question answering, which are essential for capturing complex interactions between narrative information and temporal patterns. To bridge this gap, we introduce Multimodal Time Series Benchmark (MTBench), a large-scale benchmark designed to evaluate large language models (LLMs) on time series and text understanding across financial and weather domains. MTbench comprises paired time series and textual data, including financial news with corresponding stock price movements and weather reports aligned with historical temperature records. Unlike existing benchmarks that focus on isolated modalities, MTbench provides a comprehensive testbed for models to jointly reason over structured numerical trends and unstructured textual narratives. The richness of MTbench enables formulation of diverse tasks that require a deep understanding of both text and time-series data, including time-series forecasting, semantic and technical trend analysis, and news-driven question answering (QA). These tasks target the model's ability to capture temporal dependencies, extract key insights from textual context, and integrate cross-modal information. We evaluate state-of-the-art LLMs on MTbench, analyzing their effectiveness in modeling the complex relationships between news narratives and temporal patterns. Our findings reveal significant challenges in current models, including difficulties in capturing long-term dependencies, interpreting causality in financial and weather trends, and effectively fusing multimodal information.