# UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization

arXiv: https://arxiv.org/abs/2407.03525 · HTML: https://mlsanity.com/paper/2407.03525/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [SituatedQA: Incorporating Extra-Linguistic Contexts into QA](https://mlsanity.com/paper/2109.06157/) (2021)
- ["Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding](https://mlsanity.com/paper/1909.03065/) (2019)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples](https://mlsanity.com/paper/2305.15269/) (2023)
- [Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap](https://mlsanity.com/paper/2402.19450/) (2024)
- [Reasoning about Goals, Steps, and Temporal Ordering with WikiHow](https://mlsanity.com/paper/2009.07690/) (2020)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [Temporal Reasoning on Implicit Events from Distant Supervision](https://mlsanity.com/paper/2010.12753/) (2020)
- [TEQUILA: Temporal Question Answering over Knowledge Bases](https://mlsanity.com/paper/1908.03650/) (2019)
- [TempTabQA: Temporal Question Answering for Semi-Structured Tables](https://mlsanity.com/paper/2311.08002/) (2023)
- [A Structured Learning Approach to Temporal Relation Extraction](https://mlsanity.com/paper/1906.04943/) (2019)
- [Set the Clock: Temporal Alignment of Pretrained Language Models](https://mlsanity.com/paper/2402.16797/) (2024)
- [Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?](https://mlsanity.com/paper/2406.09072/) (2024)
- [Probing Language Models for Understanding of Temporal Expressions](https://mlsanity.com/paper/2110.01113/) (2021)
- [PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answering](https://mlsanity.com/paper/2402.11034/) (2024)
- [More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering](https://mlsanity.com/paper/2311.09782/) (2023)

## Built on by (most cited first, 17 of 17)

- [Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation](https://mlsanity.com/paper/2502.17521/) (2025)
- [LLM-Safety Evaluations Lack Robustness](https://mlsanity.com/paper/2503.02574/) (2025)
- [It's High Time: A Survey of Temporal Question Answering](https://mlsanity.com/paper/2505.20243/) (2025)
- [A Study into Investigating Temporal Robustness of LLMs](https://mlsanity.com/paper/2503.17073/) (2025)
- [TSAQA: Time Series Analysis Question And Answering Benchmark](https://mlsanity.com/paper/2601.23204/) (2026)
- [MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models](https://mlsanity.com/paper/2510.19457/) (2025)
- [mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.24073/) (2025)
- [DailyQA: A Benchmark to Evaluate Web Retrieval Augmented LLMs Based on Capturing Real-World Changes](https://mlsanity.com/paper/2505.17162/) (2025)
- [TiMoE: Time-Aware Mixture of Language Experts](https://mlsanity.com/paper/2508.08827/) (2025)
- [Right Knowledge, Wrong Answer: Characterizing Parametric Temporal Conflict in Open-Weight Language Models](https://mlsanity.com/paper/2606.20959/) (2026)
- [If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs](https://mlsanity.com/paper/2511.04432/) (2025)
- [A Decentralized Retrieval Augmented Generation System with Source Reliabilities Secured on Blockchain](https://mlsanity.com/paper/2511.07577/) (2025)
- [Exposing the Illusion of Erasure in Knowledge Editing for LLMs](https://mlsanity.com/paper/2606.23276/) (2026)
- [SAGE: SLO-Aware Adaptive Retrieval for Production RAG Systems](https://mlsanity.com/paper/2608.08237/) (2026)
- [Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding](https://mlsanity.com/paper/2608.08512/) (2026)
- [Harnessing Temporal Databases for Systematic Evaluation of Factual Time-Sensitive Question-Answering in Large Language Models](https://mlsanity.com/paper/2508.02045/) (2025)
- [IndustryAssetEQA: A Neurosymbolic Operational Intelligence System for Embodied Question Answering in Industrial Asset Maintenance](https://mlsanity.com/paper/2604.23446/) (2026)

## Abstract (arXiv)

This paper introduces UnSeenTimeQA, a novel data contamination-free time-sensitive question-answering (TSQA) benchmark. It differs from existing TSQA benchmarks by avoiding web-searchable queries grounded in the real world. We present a series of time-sensitive event scenarios based on synthetically generated facts. It requires large language models (LLMs) to engage in genuine temporal reasoning without depending on the factual knowledge acquired during the pre-training phase. Our data generation framework enables on-demand generation of new samples, mitigating the risk of data leakage. We designed three types of time-sensitive questions to test LLMs' temporal reasoning abilities over sequential and parallel event occurrences. Our evaluation of five LLMs on synthetic fact-based TSQA reveals mixed results: while they perform well on simpler subsets, their overall performance remains inferior as compared to real world fact-based TSQA. Error analysis indicates that LLMs face difficulties in reasoning over long-range event dependencies and parallel events.