# Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models

arXiv: https://arxiv.org/abs/2306.08952 · HTML: https://mlsanity.com/paper/2306.08952/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- [Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework](https://mlsanity.com/paper/2305.03268/) (2023)
- [StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models](https://mlsanity.com/paper/2205.11388/) (2022)
- [Tensor Decompositions for temporal knowledge base completion](https://mlsanity.com/paper/2004.04926/) (2020)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [Reasoning Implicit Sentiment with Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.11255/) (2023)
- [Complex Temporal Question Answering on Knowledge Graphs](https://mlsanity.com/paper/2109.08935/) (2021)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction](https://mlsanity.com/paper/1909.05360/) (2019)
- [TEQUILA: Temporal Question Answering over Knowledge Bases](https://mlsanity.com/paper/1908.03650/) (2019)
- [TempoQR: Temporal Question Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2112.05785/) (2021)
- [Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphs](https://mlsanity.com/paper/2203.00255/) (2022)
- [Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events](https://mlsanity.com/paper/2104.07635/) (2021)
- [A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach](https://mlsanity.com/paper/2211.10018/) (2022)
- [Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answering](https://mlsanity.com/paper/2306.06779/) (2023)

## Built on by (most cited first, 30 of 37)

- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning](https://mlsanity.com/paper/2405.14170/) (2024)
- [TRAM: Benchmarking Temporal Reasoning for Large Language Models](https://mlsanity.com/paper/2310.00835/) (2023)
- [RAR-b: Reasoning as Retrieval Benchmark](https://mlsanity.com/paper/2404.06347/) (2024)
- [VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs](https://mlsanity.com/paper/2409.20365/) (2024)
- [Chain-of-History Reasoning for Temporal Knowledge Graph Forecasting](https://mlsanity.com/paper/2402.14382/) (2024)
- [MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2310.05157/) (2023)
- [See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses](https://mlsanity.com/paper/2408.08978/) (2024)
- [TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questions](https://mlsanity.com/paper/2502.21024/) (2025)
- [Episodic Memories Generation and Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2501.13121/) (2025)
- [A Theory for Length Generalization in Learning to Reason](https://mlsanity.com/paper/2404.00560/) (2024)
- [MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset](https://mlsanity.com/paper/2406.02106/) (2024)
- [An Overview Of Temporal Commonsense Reasoning and Acquisition](https://mlsanity.com/paper/2308.00002/) (2023)
- [A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting](https://mlsanity.com/paper/2407.11638/) (2024)
- [DyG-RAG: Dynamic Graph Retrieval-Augmented Generation with Event-Centric Reasoning](https://mlsanity.com/paper/2507.13396/) (2025)
- [Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling](https://mlsanity.com/paper/2510.01025/) (2025)
- [Conditions for Length Generalization in Learning Reasoning Skills](https://mlsanity.com/paper/2311.16173/) (2023)
- [MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models](https://mlsanity.com/paper/2510.19457/) (2025)
- [When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?](https://mlsanity.com/paper/2602.04755/) (2026)
- [ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models](https://mlsanity.com/paper/2505.19533/) (2025)
- [From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills](https://mlsanity.com/paper/2607.01456/) (2026)
- [USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents](https://mlsanity.com/paper/2505.17572/) (2025)
- [Benchmarking Spatiotemporal Reasoning in LLMs and Reasoning Models: Capabilities and Challenges](https://mlsanity.com/paper/2505.11618/) (2025)
- [In-context Interference in Chat-based Large Language Models](https://mlsanity.com/paper/2309.12727/) (2023)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [RASTeR: Robust, Agentic, and Structured Temporal Reasoning](https://mlsanity.com/paper/2406.19538/) (2024)
- [RARe: Retrieval Augmented Retrieval with In-Context Examples](https://mlsanity.com/paper/2410.20088/) (2024)
- [NewsEdits 2.0: Learning the Intentions Behind Updating News](https://mlsanity.com/paper/2411.18811/) (2024)
- [Rule Synergy Analysis using LLMs: State of the Art and Implications](https://mlsanity.com/paper/2508.19484/) (2025)
- [If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs](https://mlsanity.com/paper/2511.04432/) (2025)

## Abstract (arXiv)

Reasoning about time is of fundamental importance. Many facts are time-dependent. For example, athletes change teams from time to time, and different government officials are elected periodically. Previous time-dependent question answering (QA) datasets tend to be biased in either their coverage of time spans or question types. In this paper, we introduce a comprehensive probing dataset tempreason to evaluate the temporal reasoning capability of large language models. Our dataset includes questions of three temporal reasoning levels. In addition, we also propose a novel learning framework to improve the temporal reasoning capability of large language models, based on temporal span extraction and time-sensitive reinforcement learning. We conducted experiments in closed book QA, open book QA, and reasoning QA settings and demonstrated the effectiveness of our approach. Our code and data are released on https://github.com/DAMO-NLP-SG/TempReason.