# When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation

arXiv: https://arxiv.org/abs/2510.07238 · HTML: https://mlsanity.com/paper/2510.07238/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [Time Waits for No One! Analysis and Challenges of Temporal Misalignment](https://mlsanity.com/paper/2111.07408/) (2021)
- [Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges](https://mlsanity.com/paper/2502.12378/) (2025)
- [Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About It](https://mlsanity.com/paper/2506.23864/) (2025)
- [WINELL: Wikipedia Never-Ending Updating with LLM Agents](https://mlsanity.com/paper/2508.03728/) (2025)

## Built on by (most cited first, 6 of 6)

- [Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding](https://mlsanity.com/paper/2608.08512/) (2026)
- [Skill-based Agentic Evaluation for Real-time Data Science Tasks](https://mlsanity.com/paper/2609.16487/) (2026)
- [Replication Without Persistence in Hosted LLMs: Measurement Sensitivity in Action-Time Belief Evaluation](https://mlsanity.com/paper/2609.22478/) (2026)
- [What Should World Models Forget? Stratified Retention for Continual Adaptation](https://mlsanity.com/paper/2610.03713/) (2026)
- [Fresh at Decision, Stale at Action: Routing New Information to LLM Decisions Already Made](https://mlsanity.com/paper/2610.18126/) (2026)
- [ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Models](https://mlsanity.com/paper/2610.20472/) (2026)

## Abstract (arXiv)

The rapid evolution of large language models (LLMs) and the real world has outpaced the static nature of widely used evaluation benchmarks, raising concerns about their reliability for evaluating LLM factuality. While substantial works continue to rely on the popular but old benchmarks, their temporal misalignment with real-world facts and modern LLMs, and their effects on LLM factuality evaluation remain underexplored. Therefore, in this work, we present a systematic investigation of this issue by examining five popular factuality benchmarks and eight LLMs released across different years. An up-to-date fact retrieval pipeline and three metrics are tailored to quantify benchmark aging and its impact on LLM factuality evaluation. Experimental results and analysis illustrate that a considerable portion of samples in the widely used factuality benchmarks are outdated, leading to unreliable assessments of LLM factuality. We hope our work can provide a testbed to assess the reliability of a benchmark for LLM factuality evaluation and inspire more research on the benchmark aging issue. Codes are available in https://github.com/JiangXunyi/BenchAge.