# TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios

arXiv: https://arxiv.org/abs/2505.12891 · HTML: https://mlsanity.com/paper/2505.12891/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 1 of its 60 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 1 of 1)

- [SCTc-TE: A Comprehensive Formulation and Benchmark for Temporal Event Forecasting](https://mlsanity.com/paper/2312.01052/) (2023)

## Built on by (most cited first, 14 of 14)

- [Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents](https://mlsanity.com/paper/2512.20092/) (2025)
- [When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?](https://mlsanity.com/paper/2602.04755/) (2026)
- [TEMPO: A Realistic Multi-Domain Benchmark for Temporal Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2601.09523/) (2026)
- [CryptoBench: A Dynamic Benchmark for Expert-Level Evaluation of LLM Agents in Cryptocurrency](https://mlsanity.com/paper/2512.00417/) (2025)
- [MedMCP-Calc: Benchmarking LLMs for Realistic Medical Calculator Scenarios via MCP Integration](https://mlsanity.com/paper/2601.23049/) (2026)
- [TiMoE: Time-Aware Mixture of Language Experts](https://mlsanity.com/paper/2508.08827/) (2025)
- [Plan of Knowledge: Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering](https://mlsanity.com/paper/2511.04072/) (2025)
- [HauntAttack: When Attack Follows Reasoning as a Shadow](https://mlsanity.com/paper/2506.07031/) (2025)
- [If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs](https://mlsanity.com/paper/2511.04432/) (2025)
- [Verifier-Induced Support Reshaping in On-Policy Optimization](https://mlsanity.com/paper/2608.00220/) (2026)
- [SPAN: Benchmarking and Improving Cross-Calendar Temporal Reasoning of Large Language Models](https://mlsanity.com/paper/2511.09993/) (2025)
- [A Systematic Approach for Large Language Models Debugging](https://mlsanity.com/paper/2604.23027/) (2026)
- [LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models](https://mlsanity.com/paper/2606.25402/) (2026)
- [Chronofy: A Temporal-Logical Decay Architecture for Information Validity in Time-Aware Retrieval-Augmented Generation](https://mlsanity.com/paper/2607.20560/) (2026)

## Abstract (arXiv)

Temporal reasoning is pivotal for Large Language Models (LLMs) to comprehend the real world. However, existing works neglect the real-world challenges for temporal reasoning: (1) intensive temporal information, (2) fast-changing event dynamics, and (3) complex temporal dependencies in social interactions. To bridge this gap, we propose a multi-level benchmark TIME, designed for temporal reasoning in real-world scenarios. TIME consists of 38,522 QA pairs, covering 3 levels with 11 fine-grained sub-tasks. This benchmark encompasses 3 sub-datasets reflecting different real-world challenges: TIME-Wiki, TIME-News, and TIME-Dial. We conduct extensive experiments on reasoning models and non-reasoning models. And we conducted an in-depth analysis of temporal reasoning performance across diverse real-world scenarios and tasks, and summarized the impact of test-time scaling on temporal reasoning capabilities. Additionally, we release TIME-Lite, a human-annotated subset to foster future research and standardized evaluation in temporal reasoning. The code is available at https://github.com/sylvain-wei/TIME , the dataset is available at https://huggingface.co/datasets/SylvainWei/TIME , and the project page link is https://sylvain-wei.github.io/TIME/ .