mlsanityData to 14 Oct 2026

TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models

arXiv 2311.17667 ↗First posted 29 Nov 2023cs.CL cs.AIcited by 39

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Grasping the concept of time is a fundamental facet of human cognition, indispensable for truly comprehending the intricacies of the world. Previous studies typically focus on specific aspects of time, lacking a comprehensive temporal reasoning benchmark. To address this, we propose TimeBench, a comprehensive hierarchical temporal reasoning benchmark that covers a broad spectrum of temporal reasoning phenomena. TimeBench provides a thorough evaluation for investigating the temporal reasoning capabilities of large language models. We conduct extensive experiments on GPT-4, LLaMA2, and other popular LLMs under various settings. Our experimental results indicate a significant performance gap between the state-of-the-art LLMs and humans, highlighting that there is still a considerable distance to cover in temporal reasoning. Besides, LLMs exhibit capability discrepancies across different reasoning categories. Furthermore, we thoroughly analyze the impact of multiple aspects on temporal reasoning and emphasize the associated challenges. We aspire for TimeBench to serve as a comprehensive benchmark, fostering research in temporal reasoning. Resources are available at: https://github.com/zchuz/TimeBench

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 59 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

14 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Models2 Oct

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2023Mistral 7Bcited by 3,164
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  9. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  10. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  11. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  12. 2023Are Large Language Models Temporally Grounded?cited by 27
1 more
  1. 2023GLoRE: Evaluating Logical Reasoning of Large Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 19; 2026: 12.

39 papers.

Sort:

By area:

  1. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  2. 2024Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoningcited by 41
  3. 2024Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?cited by 25
  4. 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMscited by 25
  5. 2026TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Taskscited by 17
  6. 2025REALM-Bench: A Benchmark for Evaluating Multi-Agent Systems on Real-world, Dynamic Planning and Scheduling Taskscited by 16
  7. 2025TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questionscited by 15
  8. 2024Temporal Knowledge Graph Question Answering: A Surveycited by 9
  9. 2024Rethinking Emotion Annotations in the Era of Large Language Modelscited by 8
  10. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 8
  11. 2025ExAnte: A Benchmark for Ex-Ante Inference in Large Language Modelscited by 6
  12. 2024Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narrativescited by 5
27 more
  1. 2025ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planningcited by 5
  2. 2025USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agentscited by 5
  3. 2024LTLBench: Towards Benchmarks for Evaluating Temporal Reasoning in Large Language Modelscited by 4
  4. 2025Too Easily Fooled? Prompt Injection Breaks LLMs on Frustratingly Simple Multiple-Choice Questionscited by 4
  5. 2025TRAVELER: A Benchmark for Evaluating Temporal Reasoning across Vague, Implicit and Explicit Referencescited by 3
  6. 2025PMOA-TTS: Introducing the PubMed Open Access Textual Times Series Corpuscited by 3
  7. 2025ALAS: Transactional and Dynamic Multi-Agent LLM Planningcited by 3
  8. 2024RASTeR: Robust, Agentic, and Structured Temporal Reasoningcited by 2
  9. 2025TiMoE: Time-Aware Mixture of Language Expertscited by 2
  10. 2025Critical Confabulation: Can LLMs Hallucinate for Social Good?cited by 2
  11. 2025SNOW: Spatio-Temporal Scene Understanding with World Knowledge for Open-World Embodied Reasoningcited by 2
  12. 2026Causal Stories from Sensor Traces: Auditing Epistemic Overreach in LLM-Generated Personal Sensing Explanationscited by 2
  13. 2025Self-Exploring Language Models for Explainable Link Forecasting on Temporal Graphs via Reinforcement Learningcited by 1
  14. 2025NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Modelscited by 1
  15. 2026Three Generations of Healthcare IT: From the Digital Record to the Computable Care Processcited by 1
  16. 2025LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoningcited by 0
  17. 2025Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMscited by 0
  18. 2025CaMiT: A Time-Aware Car Model Dataset for Classification and Generationcited by 0
  19. 2026Bridging Graph Structure and Knowledge-Guided Editing for Interpretable Temporal Knowledge Graph Reasoningcited by 0
  20. 2026AdapTime: Enabling Adaptive Temporal Reasoning in Large Language Modelscited by 0
  21. 2026Teaching Large Language Models When Not to Know: Learning Temporal Critique for Ex-Ante Reasoningcited by 0
  22. 2026On the Cultural Anachronism and Temporal Reasoning in Vision Language Modelscited by 0
  23. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  24. 2026PreSIST: Vision-Language-Informed Object Persistence Prediction in Open-World Scenescited by 0
  25. 2026PredAct-Bench: Benchmarking Tool-Augmented Dialogue under Controlled Tool Noisecited by 0
  26. 2026Can Large Language Models Reason about Event-Time Stream-Processing Semantics?cited by 0
  27. 2026AgentTime: Can Agents Estimate and Control Their Own Runtime?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios
  2. 2026What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
  3. 2023MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models
  4. 2024Timo: Towards Better Temporal Reasoning for Language Models
  5. 2025Discrete Minds in a Continuous World: Do Language Models Know Time Passes?
  6. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time
  7. 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events
  8. 2026Measuring Iterative Temporal Reasoning with Time Puzzles