# TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks

arXiv: https://arxiv.org/abs/2602.13272 · HTML: https://mlsanity.com/paper/2602.13272/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 75 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 30 of 32)

- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [Large Language Models for Time Series: A Survey](https://mlsanity.com/paper/2402.01801/) (2024)
- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data](https://mlsanity.com/paper/2412.11376/) (2024)
- [From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection](https://mlsanity.com/paper/2409.17515/) (2024)
- [Language Models Still Struggle to Zero-shot Reason about Time Series](https://mlsanity.com/paper/2404.11757/) (2024)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy](https://mlsanity.com/paper/2406.11678/) (2024)
- [TRAM: Benchmarking Temporal Reasoning for Large Language Models](https://mlsanity.com/paper/2310.00835/) (2023)
- [TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models](https://mlsanity.com/paper/2311.17667/) (2023)
- [TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents](https://mlsanity.com/paper/2502.11418/) (2025)
- [Spectral Temporal Graph Neural Network for Trajectory Prediction](https://mlsanity.com/paper/2106.02930/) (2021)
- [TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model](https://mlsanity.com/paper/2409.02322/) (2024)
- [TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysis](https://mlsanity.com/paper/2410.04047/) (2024)
- [Intervention-Aware Forecasting: Breaking Historical Limits from a System Perspective](https://mlsanity.com/paper/2405.13522/) (2024)
- [HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL](https://mlsanity.com/paper/2503.16528/) (2025)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [VeriMoA: A Mixture-of-Agents Framework for Spec-to-HDL Generation](https://mlsanity.com/paper/2510.27617/) (2025)
- [Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs](https://mlsanity.com/paper/2511.06134/) (2025)
- [The Causal Chambers: Real Physical Systems as a Testbed for AI Methodology](https://mlsanity.com/paper/2404.11341/) (2024)
- [When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference](https://mlsanity.com/paper/2509.01822/) (2025)
- [A Multi-scale Time-series Dataset with Benchmark for Machine Learning in Decarbonized Energy Grids](https://mlsanity.com/paper/2110.06324/) (2021)
- [Empowering Time Series Forecasting with LLM-Agents](https://mlsanity.com/paper/2508.04231/) (2025)
- [Foundation Models for Demand Forecasting via Dual-Strategy Ensembling](https://mlsanity.com/paper/2507.22053/) (2025)
- [TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis](https://mlsanity.com/paper/2510.06063/) (2025)

## Built on by (most cited first, 17 of 17)

- [Auditing Multi-Agent LLM Reasoning Trees Outperforms Majority Vote and LLM-as-Judge](https://mlsanity.com/paper/2602.09341/) (2026)
- [Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning](https://mlsanity.com/paper/2603.07972/) (2026)
- [RaMem: Contextual Reinstatement for Long-term Agentic Memory](https://mlsanity.com/paper/2606.22844/) (2026)
- [TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning](https://mlsanity.com/paper/2606.01498/) (2026)
- [TimeClaw: A Time-Series AI Agent with Exploratory Execution Learning](https://mlsanity.com/paper/2605.10038/) (2026)
- [AION: Next-Generation Tasks and Practical Harness for Time Series](https://mlsanity.com/paper/2605.25045/) (2026)
- [Dr-CiK: A Benchmark testing Deep Research for Context-Aided Forecasting](https://mlsanity.com/paper/2605.27904/) (2026)
- [Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context](https://mlsanity.com/paper/2609.15087/) (2026)
- [LLM Agents for Time-Series: A Survey](https://mlsanity.com/paper/2608.26226/) (2026)
- [TraceBench: Controlled Evaluation of LLM Agents for Time-Series Root-Cause Attribution](https://mlsanity.com/paper/2608.27182/) (2026)
- [Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting](https://mlsanity.com/paper/2606.26487/) (2026)
- [WaveformQA: Benchmarking LLM Temporal Reasoning on Digital Waveforms](https://mlsanity.com/paper/2607.20638/) (2026)
- [TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments](https://mlsanity.com/paper/2608.14270/) (2026)
- [TimeLitmus: A Diagnostic Benchmark for Cross-Modal Understanding and Explanation Faithfulness in Event-Conditioned Time-Series Prediction](https://mlsanity.com/paper/2609.24677/) (2026)
- [TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent](https://mlsanity.com/paper/2609.27277/) (2026)
- [Forecast Workflow Bench: Evaluating Language-Model Decisions with Budgeted Forecast Tools](https://mlsanity.com/paper/2609.27385/) (2026)
- [GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation](https://mlsanity.com/paper/2610.17794/) (2026)

## Abstract (arXiv)

It is unclear whether strong forecasting performance reflects genuine temporal understanding or the ability to reason under contextual and event-driven conditions. We introduce TemporalBench, a multi-domain benchmark designed to evaluate temporal reasoning behavior under progressively richer informational settings. TemporalBench adopts a four-tier task taxonomy that examines historical structure interpretation, context-free forecasting, contextual temporal reasoning, and event-conditioned prediction across four real-world domains: retail, healthcare, energy, and physical systems. By controlling access to future targets and contextual information, the benchmark enables a diagnostic analysis of whether models can correctly interpret temporal patterns, align them with external context, and adapt predictions when conditions change. Extensive baseline experiments show that strong numerical forecasting accuracy does not reliably translate into robust contextual or event-aware temporal reasoning; instead, existing agent frameworks exhibit fragmented strengths and systematic failure modes that remain largely hidden under forecasting-only benchmarks. The TemporalBench dataset is publicly available at https://huggingface.co/datasets/Melady/TemporalBench, and we additionally provide a public leaderboard at https://huggingface.co/spaces/Melady/TemporalBench_Leaderboard.