mlsanityData to 7 Oct 2026

UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization

arXiv 2407.03525 ↗First posted 3 Jul 2024cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

This paper introduces UnSeenTimeQA, a novel data contamination-free time-sensitive question-answering (TSQA) benchmark. It differs from existing TSQA benchmarks by avoiding web-searchable queries grounded in the real world. We present a series of time-sensitive event scenarios based on synthetically generated facts. It requires large language models (LLMs) to engage in genuine temporal reasoning without depending on the factual knowledge acquired during the pre-training phase. Our data generation framework enables on-demand generation of new samples, mitigating the risk of data leakage. We designed three types of time-sensitive questions to test LLMs' temporal reasoning abilities over sequential and parallel event occurrences. Our evaluation of five LLMs on synthetic fact-based TSQA reveals mixed results: while they perform well on simpler subsets, their overall performance remains inferior as compared to real world fact-based TSQA. Error analysis indicates that LLMs face difficulties in reasoning over long-range event dependencies and parallel events.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

31 of 47 references.

References · 31

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  7. 2018Datasheets for Datasetscited by 1,371
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  10. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  11. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  12. 2022RealTime QA: What's the Answer Right Now?cited by 239
19 more
  1. 2021A Dataset for Answering Time-Sensitive Questionscited by 150
  2. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  3. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 121
  4. 2021Question Answering Over Temporal Knowledge Graphscited by 99
  5. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  6. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
  7. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  8. 2020Reasoning about Goals, Steps, and Temporal Ordering with WikiHowcited by 66
  9. 2020TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questionscited by 65
  10. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  11. 2019TEQUILA: Temporal Question Answering over Knowledge Basescited by 47
  12. 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
  13. 2024Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?cited by 25
  14. 2019A Structured Learning Approach to Temporal Relation Extractioncited by 24
  15. 2024Set the Clock: Temporal Alignment of Pretrained Language Modelscited by 24
  16. 2021Probing Language Models for Understanding of Temporal Expressionscited by 16
  17. 2024PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answeringcited by 15
  18. 2023More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineeringcited by 13
  19. 2024ActionReasoningBench: Reasoning about Actions with and without Ramification Constraintscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  2. 2025LLM-Safety Evaluations Lack Robustnesscited by 16
  3. 2025It's High Time: A Survey of Temporal Question Answeringcited by 12
  4. 2025A Study into Investigating Temporal Robustness of LLMscited by 10
  5. 2026TSAQA: Time Series Analysis Question And Answering Benchmarkcited by 8
  6. 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
  7. 2025mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generationcited by 4
  8. 2025DailyQA: A Benchmark to Evaluate Web Retrieval Augmented LLMs Based on Capturing Real-World Changescited by 3
  9. 2025TiMoE: Time-Aware Mixture of Language Expertscited by 2
  10. 2026Right Knowledge, Wrong Answer: Characterizing Parametric Temporal Conflict in Open-Weight Language Modelscited by 2
  11. 2025If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMscited by 1
  12. 2025A Decentralized Retrieval Augmented Generation System with Source Reliabilities Secured on Blockchaincited by 1
5 more
  1. 2026Exposing the Illusion of Erasure in Knowledge Editing for LLMscited by 1
  2. 2026SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systemscited by 1
  3. 2026Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understandingcited by 1
  4. 2025Harnessing Temporal Databases for Systematic Evaluation of Factual Time-Sensitive Question-Answering in Large Language Modelscited by 0
  5. 2026IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Temporal Question-Answering Capabilities of Large Language Models Over Anonymized Data
  2. 2024Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering
  3. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering
  4. 2022A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases
  5. 2023Question Answering as Programming for Solving Time-Sensitive Questions
  6. 2023Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic
  7. 2025LLM-Symbolic Integration for Robust Temporal Tabular Reasoning
  8. 2023Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning