mlsanityData to 14 Oct 2026

EvoWiki: Evaluating LLMs on Evolving Knowledge

arXiv 2412.13582 ↗First posted 18 Dec 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Knowledge utilization is a critical aspect of LLMs, and understanding how they adapt to evolving knowledge is essential for their effective deployment. However, existing benchmarks are predominantly static, failing to capture the evolving nature of LLMs and knowledge, leading to inaccuracies and vulnerabilities such as contamination. In this paper, we introduce EvoWiki, an evolving dataset designed to reflect knowledge evolution by categorizing information into stable, evolved, and uncharted states. EvoWiki is fully auto-updatable, enabling precise evaluation of continuously changing knowledge and newly released LLMs. Through experiments with Retrieval-Augmented Generation (RAG) and Contunual Learning (CL), we evaluate how effectively LLMs adapt to evolving knowledge. Our results indicate that current models often struggle with evolved knowledge, frequently providing outdated or incorrect responses. Moreover, the dataset highlights a synergistic effect between RAG and CL, demonstrating their potential to better adapt to evolving knowledge. EvoWiki provides a robust benchmark for advancing future research on the knowledge evolution capabilities of large language models.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

30 of 40 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  8. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  10. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  11. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  12. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
18 more
  1. 2022RealTime QA: What's the Answer Right Now?cited by 244
  2. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 204
  3. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  4. 2021A Dataset for Answering Time-Sensitive Questionscited by 154
  5. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 119
  6. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 108
  7. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  8. 2021Question Answering Over Temporal Knowledge Graphscited by 99
  9. 2020SCATTER: Selective Context Attentional Scene Text Recognizercited by 69
  10. 2021Complex Temporal Question Answering on Knowledge Graphscited by 69
  11. 2024Instruction-tuned Language Models are Better Knowledge Learnerscited by 55
  12. 2019TEQUILA: Temporal Question Answering over Knowledge Basescited by 47
  13. 2024DYNAMICQA: Tracing Internal Knowledge Conflicts in Language Modelscited by 34
  14. 2024Context versus Prior Knowledge in Language Modelscited by 26
  15. 2023Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Datacited by 23
  16. 2023Intuitive or Dependent? Investigating LLMs' Behavior Style to Conflicting Promptscited by 23
  17. 2024PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answeringcited by 15
  18. 2024A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potentialcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 28
  2. 2025When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorationscited by 7
  3. 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
  4. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  5. 2026MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledgecited by 5
  6. 2025ScienceMeter: Tracking Scientific Knowledge Updates in Language Modelscited by 4
  7. 2026OpenSkillEval: Automatically Auditing the Open Skill Ecosystem for LLM Agentscited by 4
  8. 2025When Facts Change: Probing LLMs on Evolving Knowledge with evolveQAcited by 2
  9. 2026Large Language Models Lack Temporal Awareness of Medical Knowledgecited by 2
  10. 2026Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understandingcited by 1
  11. 2025Aligning Language Models with Real-time Knowledge Editingcited by 0
  12. 2025Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statementscited by 0
1 more
  1. 2026ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real World
  2. 2025Memorization vs. Reasoning: Updating LLMs with New Knowledge
  3. 2025KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning
  4. 2026EvoRAG: Making Knowledge Graph-based RAG Automatically Evolve through Feedback-driven Backpropagation
  5. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
  6. 2024GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge?
  7. 2025Towards Understanding How Knowledge Evolves in Large Vision-Language Models
  8. 2024Evolving Knowledge Distillation with Large Language Models and Active Learning