# TiEBe: Tracking Language Model Recall of Notable Worldwide Events Through Time

arXiv: https://arxiv.org/abs/2501.07482 · HTML: https://mlsanity.com/paper/2501.07482/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 13 Jan 2025.

## Builds on (most cited first, 20 of 20)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://mlsanity.com/paper/2406.09948/) (2024)
- [Continual Learning for Large Language Models: A Survey](https://mlsanity.com/paper/2402.01364/) (2024)
- [Simple and Scalable Strategies to Continually Pre-train Large Language Models](https://mlsanity.com/paper/2403.08763/) (2024)
- [TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models](https://mlsanity.com/paper/2204.14211/) (2022)
- [Investigating the Catastrophic Forgetting in Multimodal Large Language Models](https://mlsanity.com/paper/2309.10313/) (2023)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [Low-resource Languages: A Review of Past Work and Future Challenges](https://mlsanity.com/paper/2006.07264/) (2020)
- [CITB: A Benchmark for Continual Instruction Tuning](https://mlsanity.com/paper/2310.14510/) (2023)
- [Quantifying the Dialect Gap and its Correlates Across Languages](https://mlsanity.com/paper/2310.15135/) (2023)
- [Continual Learning Under Language Shift](https://mlsanity.com/paper/2311.01200/) (2023)
- [Sabi'a-3 Technical Report](https://mlsanity.com/paper/2410.12049/) (2024)
- [LLeMpower: Understanding Disparities in the Control and Access of Large Language Models](https://mlsanity.com/paper/2404.09356/) (2024)

## Built on by (most cited first, 8 of 8)

- [Ticket-Bench: A Kickoff for Multilingual and Regionalized Agent Evaluation](https://mlsanity.com/paper/2509.14477/) (2025)
- [PoETa v2: Toward More Robust Evaluation of Large Language Models in Portuguese](https://mlsanity.com/paper/2511.17808/) (2025)
- [BLUEX Revisited: Enhancing Benchmark Coverage with Automatic Captioning](https://mlsanity.com/paper/2508.21294/) (2025)
- [LocalBench: Benchmarking LLMs on County-Level Local Knowledge and Reasoning](https://mlsanity.com/paper/2511.10459/) (2025)
- [Frame-Guided Synthetic Claim Generation for Automatic Fact-Checking Using High-Volume Tabular Data](https://mlsanity.com/paper/2601.17232/) (2026)
- [Synthetic Rewriting as a Quality Multiplier: Evidence from Portuguese Continued Pretraining](https://mlsanity.com/paper/2603.24826/) (2026)
- [MARCA: A Checklist-Based Benchmark for Multilingual Web Search](https://mlsanity.com/paper/2604.14448/) (2026)
- [MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models](https://mlsanity.com/paper/2609.38543/) (2026)

## Abstract (arXiv)

As the knowledge landscape evolves and large language models (LLMs) become increasingly widespread, there is a growing need to keep these models updated with current events. While existing benchmarks assess general factual recall, few studies explore how LLMs retain knowledge over time or across different regions. To address these gaps, we present the Timely Events Benchmark (TiEBe), a dataset of over 23,000 question-answer pairs centered on notable global and regional events, spanning more than 10 years of events, 23 regions, and 13 languages. TiEBe leverages structured retrospective data from Wikipedia to identify notable events through time. These events are then used to construct a benchmark to evaluate LLMs' understanding of global and regional developments, grounded in factual evidence beyond Wikipedia itself. Our results reveal significant geographic disparities in factual recall, emphasizing the need for more balanced global representation in LLM training. We also observe a Pearson correlation of more than 0.7 between models' performance in TiEBe and various countries' socioeconomic indicators, such as HDI. In addition, we examine the impact of language on factual recall by posing questions in the native language of the region where each event occurred, uncovering substantial performance gaps for low-resource languages.