# MMTS-BENCH: A Comprehensive Benchmark for Time Series Understanding and Reasoning

arXiv: https://arxiv.org/abs/2602.08588 · HTML: https://mlsanity.com/paper/2602.08588/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2026.

## Built on by (most cited first, 9 of 9)

- [CLIR-Bench: Benchmarking Multimodal Question Answering over Irregular Clinical Time Series](https://mlsanity.com/paper/2607.09880/) (2026)
- [LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics](https://mlsanity.com/paper/2604.17295/) (2026)
- [AION: Next-Generation Tasks and Practical Harness for Time Series](https://mlsanity.com/paper/2605.25045/) (2026)
- [TS-Skill: A Benchmark for Evaluating Analytical Skills in Time-Series Question Answering](https://mlsanity.com/paper/2605.24703/) (2026)
- [T-SMART: Mechanism-Level Attribution for Tool-Augmented Time-Series Question Answering](https://mlsanity.com/paper/2609.14142/) (2026)
- [TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent](https://mlsanity.com/paper/2609.27277/) (2026)
- [Zero-Shot Time-Series Question Answering via Decoupled Perception and Reasoning](https://mlsanity.com/paper/2610.04942/) (2026)
- [GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation](https://mlsanity.com/paper/2610.17794/) (2026)
- [MMTA: Benchmarking Multimodal Temporal Analysis with Time Series, Text, and Vision](https://mlsanity.com/paper/2610.17856/) (2026)

## Abstract (arXiv)

Time series data are central to domains such as finance, healthcare, and cloud computing, yet existing benchmarks for evaluating various large language models (LLMs) on temporal tasks remain scattered and unsystematic. To bridge this gap, we introduce MMTS-BENCH, a comprehensive multimodal benchmark built upon a hierarchical taxonomy of time-series tasks, spanning structural awareness, feature analysis, temporal reasoning, sequence matching and cross-modal alignment. MMTS-BENCH comprises 2,424 time series question answering (TSQA) pairs across 4 subsets: Base, InWild, Match, and Align, generated through a progressive real-world QA framework and modular synthetic data construction. We conduct extensive evaluations on closed-source, open-source LLMs and existing time series adapted large language models (TS-LLMs), revealing that: (1) TS-LLMs significantly lag behind general-purpose LLMs in cross-domain generalization, (2) LLMs show weaknesses in local tasks compared to global tasks, (3) chain-of-thought (CoT) reasoning and multimodal integration substantially improve performance, and (4) the dominant factor in existing TS-LLMs remains the backbone network capability rather than the time series encoder design. MMTS-BENCH not only provides a rigorous evaluation framework but also offers clear directions for advancing LLMs toward robust, interpretable, and generalizable time-series reasoning.