mlsanityData to 7 Oct 2026

Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning

arXiv 2311.09821 ↗First posted 16 Nov 2023cs.CLcited by 18

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Knowledge in the real world is being updated constantly. However, it is costly to frequently update large language models (LLMs). Therefore, it is crucial for LLMs to understand the concept of temporal knowledge. However, prior works on temporal question answering (TQA) did not emphasize multi-answer and multi-hop types of temporal reasoning. In this paper, we propose a complex temporal question-answering dataset Complex-TR that focuses on multi-answer and multi-hop temporal reasoning. Besides, we also propose a novel data augmentation strategy to improve the complex temporal reasoning capability and robustness of LLMs. We conducted experiments on multiple temporal QA datasets. Experimental results show that our method is able to improve LLMs' performance on temporal QA benchmarks by significant margins. Our code and data are released at: https://github.com/nusnlp/complex-tr.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 42 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

7 of 42 references.

References · 7

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  4. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  5. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  6. 2021MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NERcited by 42
  7. 2022QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 6.

18 papers.

Sort:

By area:

  1. 2024Large Language Models Can Learn Temporal Reasoningcited by 110
  2. 2024Timo: Towards Better Temporal Reasoning for Language Modelscited by 21
  3. 2026REMem: Reasoning with Episodic Memory in Language Agentcited by 20
  4. 2025It's High Time: A Survey of Temporal Question Answeringcited by 12
  5. 2024Enhancing Temporal Understanding in LLMs for Semi-structured Tablescited by 8
  6. 2025DyG-RAG: Dynamic Graph Retrieval-Augmented Generation with Event-Centric Reasoningcited by 8
  7. 2025Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoningcited by 6
  8. 2025Around the World in 24 Hours: Probing LLM Knowledge of Time and Placecited by 6
  9. 2024Remember This Event That Year? Assessing Temporal Information and Reasoning in Large Language Modelscited by 3
  10. 2024DateLogicQA: Benchmarking Temporal Biases in Large Language Modelscited by 2
  11. 2025Leveraging LLM For Synchronizing Information Across Multilingual Tablescited by 2
  12. 2026BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidencecited by 2
6 more
  1. 2026Right Knowledge, Wrong Answer: Characterizing Parametric Temporal Conflict in Open-Weight Language Modelscited by 2
  2. 2025Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generationcited by 1
  3. 2025Structured yet Bounded Temporal Understanding in Large Language Modelscited by 0
  4. 2026TempViz: On the Evaluation of Temporal Knowledge in Text-to-Image Modelscited by 0
  5. 2026Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrievalcited by 0
  6. 2026ScalableRAG: High-Quality RAG at Zero Ingestion Costcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models
  2. 2025A Study into Investigating Temporal Robustness of LLMs
  3. 2023Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic
  4. 2023Temporal Knowledge Question Answering via Abstract Reasoning Induction
  5. 2026AdapTime: Enabling Adaptive Temporal Reasoning in Large Language Models
  6. 2025On the Temporal Question-Answering Capabilities of Large Language Models Over Anonymized Data
  7. 2022A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases
  8. 2025MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning