# A Study into Investigating Temporal Robustness of LLMs

arXiv: https://arxiv.org/abs/2503.17073 · HTML: https://mlsanity.com/paper/2503.17073/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- ["Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding](https://mlsanity.com/paper/1909.03065/) (2019)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation](https://mlsanity.com/paper/2202.07654/) (2022)
- [Zero-shot Temporal Relation Extraction with ChatGPT](https://mlsanity.com/paper/2304.05454/) (2023)
- [Time Masking for Temporal Language Models](https://mlsanity.com/paper/2110.06366/) (2021)
- [Dynamic Word Embeddings for Evolving Semantic Discovery](https://mlsanity.com/paper/1703.00607/) (2017)
- [ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning](https://mlsanity.com/paper/2012.15283/) (2020)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [QuanTemp: A real-world open-domain benchmark for fact-checking numerical claims](https://mlsanity.com/paper/2403.17169/) (2024)
- [Temporal Attention for Language Models](https://mlsanity.com/paper/2202.02093/) (2022)
- [BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information](https://mlsanity.com/paper/2204.13032/) (2022)
- [UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization](https://mlsanity.com/paper/2407.03525/) (2024)
- [Salient Span Masking for Temporal Understanding](https://mlsanity.com/paper/2303.12860/) (2023)
- [HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis](https://mlsanity.com/paper/2202.03612/) (2022)
- [Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoning](https://mlsanity.com/paper/2310.14709/) (2023)
- [Time-aware Prompting for Text Generation](https://mlsanity.com/paper/2211.02162/) (2022)
- [An Overview Of Temporal Commonsense Reasoning and Acquisition](https://mlsanity.com/paper/2308.00002/) (2023)

## Built on by (most cited first, 11 of 11)

- [It's High Time: A Survey of Temporal Question Answering](https://mlsanity.com/paper/2505.20243/) (2025)
- [Sample Efficient Demonstration Selection for In-Context Learning](https://mlsanity.com/paper/2506.08607/) (2025)
- [When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?](https://mlsanity.com/paper/2602.04755/) (2026)
- [TEMPO: A Realistic Multi-Domain Benchmark for Temporal Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2601.09523/) (2026)
- [RankArena: A Unified Platform for Evaluating Retrieval, Reranking and RAG with Human and LLM Feedback](https://mlsanity.com/paper/2508.05512/) (2025)
- [A Vision for Geo-Temporal Deep Research Systems: Towards Comprehensive, Transparent, and Reproducible Geo-Temporal Information Synthesis](https://mlsanity.com/paper/2506.14345/) (2025)
- [Efficient Temporal-aware Matryoshka Adaptation for Temporal Information Retrieval](https://mlsanity.com/paper/2601.05549/) (2026)
- [Do Language Models Consistently Encode the Current Year?](https://mlsanity.com/paper/2608.15507/) (2026)
- [Time to Revist Exact Match](https://mlsanity.com/paper/2509.16720/) (2025)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making](https://mlsanity.com/paper/2609.05149/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) encapsulate a surprising amount of factual world knowledge. However, their performance on temporal questions and historical knowledge is limited because they often cannot understand temporal scope and orientation or neglect the temporal aspect altogether. In this study, we aim to measure precisely how robust LLMs are for question answering based on their ability to process temporal information and perform tasks requiring temporal reasoning and temporal factual knowledge. Specifically, we design eight time-sensitive robustness tests for factual information to check the sensitivity of six popular LLMs in the zero-shot setting. Overall, we find LLMs lacking temporal robustness, especially to temporal reformulations and the use of different granularities of temporal references. We show how a selection of these eight tests can be used automatically to judge a model's temporal robustness for user questions on the fly. Finally, we apply the findings of this study to improve the temporal QA performance by up to 55 percent.