mlsanityData to 14 Oct 2026

Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models

arXiv 2306.08952 ↗First posted 15 Jun 2023cs.CL cs.AIcited by 37

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Reasoning about time is of fundamental importance. Many facts are time-dependent. For example, athletes change teams from time to time, and different government officials are elected periodically. Previous time-dependent question answering (QA) datasets tend to be biased in either their coverage of time spans or question types. In this paper, we introduce a comprehensive probing dataset tempreason to evaluate the temporal reasoning capability of large language models. Our dataset includes questions of three temporal reasoning levels. In addition, we also propose a novel learning framework to improve the temporal reasoning capability of large language models, based on temporal span extraction and time-sensitive reinforcement learning. We conducted experiments in closed book QA, open book QA, and reasoning QA settings and demonstrated the effectiveness of our approach. Our code and data are released on https://github.com/DAMO-NLP-SG/TempReason.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 55 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

28 of 55 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  11. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  12. 2022RealTime QA: What's the Answer Right Now?cited by 244
16 more
  1. 2021A Dataset for Answering Time-Sensitive Questionscited by 154
  2. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  3. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  4. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 126
  5. 2020Tensor Decompositions for temporal knowledge base completioncited by 115
  6. 2021Question Answering Over Temporal Knowledge Graphscited by 99
  7. 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 80
  8. 2021Complex Temporal Question Answering on Knowledge Graphscited by 69
  9. 2020TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questionscited by 65
  10. 2019Joint Event and Temporal Relation Extraction with Shared Representations and Structured Predictioncited by 53
  11. 2019TEQUILA: Temporal Question Answering over Knowledge Basescited by 47
  12. 2021TempoQR: Temporal Question Reasoning over Knowledge Graphscited by 26
  13. 2022Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphscited by 26
  14. 2021Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Eventscited by 22
  15. 2022A Dataset for Hyper-Relational Extraction and a Cube-Filling Approachcited by 13
  16. 2023Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answeringcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 13; 2026: 7.

37 papers.

Sort:

By area:

  1. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  2. 2024Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoningcited by 41
  3. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 40
  4. 2024RAR-b: Reasoning as Retrieval Benchmarkcited by 36
  5. 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMscited by 29
  6. 2024Chain-of-History Reasoning for Temporal Knowledge Graph Forecastingcited by 17
  7. 2023MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Modelscited by 15
  8. 2024See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknessescited by 15
  9. 2025TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questionscited by 15
  10. 2025Episodic Memories Generation and Evaluation Benchmark for Large Language Modelscited by 14
  11. 2024A Theory for Length Generalization in Learning to Reasoncited by 11
  12. 2024MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Datasetcited by 11
25 more
  1. 2023An Overview Of Temporal Commonsense Reasoning and Acquisitioncited by 10
  2. 2024A Comprehensive Evaluation of Large Language Models on Temporal Event Forecastingcited by 8
  3. 2025DyG-RAG: Dynamic Graph Retrieval-Augmented Generation with Event-Centric Reasoningcited by 8
  4. 2025Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scalingcited by 8
  5. 2023Conditions for Length Generalization in Learning Reasoning Skillscited by 7
  6. 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
  7. 2026When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?cited by 7
  8. 2025ExAnte: A Benchmark for Ex-Ante Inference in Large Language Modelscited by 6
  9. 2026From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skillscited by 6
  10. 2025USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agentscited by 5
  11. 2025Benchmarking Spatiotemporal Reasoning in LLMs and Reasoning Models: Capabilities and Challengescited by 4
  12. 2023In-context Interference in Chat-based Large Language Modelscited by 3
  13. 2025Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neurosciencecited by 3
  14. 2024RASTeR: Robust, Agentic, and Structured Temporal Reasoningcited by 2
  15. 2024RARe: Retrieval Augmented Retrieval with In-Context Examplescited by 2
  16. 2024NewsEdits 2.0: Learning the Intentions Behind Updating Newscited by 1
  17. 2025Rule Synergy Analysis using LLMs: State of the Art and Implicationscited by 1
  18. 2025If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMscited by 1
  19. 2025NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Modelscited by 1
  20. 2025Harnessing Temporal Databases for Systematic Evaluation of Factual Time-Sensitive Question-Answering in Large Language Modelscited by 0
  21. 2026Temporal-Aware Heterogeneous Graph Reasoning with Multi-View Fusion for Temporal Question Answeringcited by 0
  22. 2026Temporal Reasoning Is Not the Bottleneck: A Probabilistic Inconsistency Framework for Neuro-Symbolic QAcited by 0
  23. 2026Teaching Large Language Models When Not to Know: Learning Temporal Critique for Ex-Ante Reasoningcited by 0
  24. 2026A Study of Temporal Fusion Strategies for Named Entity Recognition in Historical Textscited by 0
  25. 2026Can Large Language Models Reason about Event-Time Stream-Processing Semantics?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic
  2. 2023Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning
  3. 2026AdapTime: Enabling Adaptive Temporal Reasoning in Large Language Models
  4. 2024Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering
  5. 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time
  6. 2024Timo: Towards Better Temporal Reasoning for Language Models
  7. 2025TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios
  8. 2026TimeThink: Eliciting Compositional Reasoning in Timeseries Large Language Models