# Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time

arXiv: https://arxiv.org/abs/2409.13338 · HTML: https://mlsanity.com/paper/2409.13338/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection](https://mlsanity.com/paper/2007.11464/) (2020)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement](https://mlsanity.com/paper/2405.20701/) (2024)

## Built on by (most cited first, 13 of 13)

- [TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records](https://mlsanity.com/paper/2503.04176/) (2025)
- [NarraBench: A Comprehensive Framework for Narrative Benchmarking](https://mlsanity.com/paper/2510.09869/) (2025)
- [Challenges and Future Directions of Data-Centric AI Alignment](https://mlsanity.com/paper/2410.01957/) (2024)
- [Temporal Alignment of Time Sensitive Facts with Activation Engineering](https://mlsanity.com/paper/2505.14158/) (2025)
- [Can LLMs Perceive Time? An Empirical Investigation](https://mlsanity.com/paper/2604.00010/) (2026)
- [TiMoE: Time-Aware Mixture of Language Experts](https://mlsanity.com/paper/2508.08827/) (2025)
- [From Demographics to Survey Anchors: Evaluating LLM Agents for Modeling Retirement Attitudes](https://mlsanity.com/paper/2605.16303/) (2026)
- [Right Knowledge, Wrong Answer: Characterizing Parametric Temporal Conflict in Open-Weight Language Models](https://mlsanity.com/paper/2606.20959/) (2026)
- [TIME: Temporally Intelligent Meta-reasoning Engine for Context-Triggered Explicit Reasoning](https://mlsanity.com/paper/2601.05300/) (2026)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [Forecasting With LLMs: Improved Generalization Through Feature Steering](https://mlsanity.com/paper/2606.27199/) (2026)
- [LexKairos: Benchmarking Legal Temporal Capabilities in LLMs](https://mlsanity.com/paper/2608.09106/) (2026)
- [Lost in Historical Time? A Polish History Matura Benchmark for Large Language Models](https://mlsanity.com/paper/2608.12343/) (2026)

## Abstract (arXiv)

Who is the US President? The answer changes depending on when the question is asked. While large language models (LLMs) are evaluated on various reasoning tasks, they often miss a crucial dimension: time. In real-world scenarios, the correctness of answers is frequently tied to temporal context. To address this gap, we present a novel framework and dataset spanning over 8,000 events from 2018 to 2024, annotated with day-level granularity and sourced globally across domains such as politics, science, and business. Our TimeShift evaluation method systematically probes LLMs for temporal reasoning, revealing that base models often outperform instruction-tuned and synthetic-trained counterparts on time-sensitive recall. Additionally, we find that even large-scale models exhibit brittleness in handling paraphrased facts, highlighting unresolved challenges in temporal consistency. By identifying these limitations, our work provides a significant step toward advancing time-aware language models capable of adapting to the dynamic nature of real-world knowledge.