mlsanityData to 14 Oct 2026

How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances

arXiv 2310.07343 ↗First posted 11 Oct 2023cs.CLcited by 33

Not on the map. Why?

Abstract (arXiv)

Although large language models (LLMs) are impressive in solving various tasks, they can quickly be outdated after deployment. Maintaining their up-to-date status is a pressing concern in the current era. This paper provides a comprehensive review of recent advances in aligning LLMs with the ever-changing world knowledge without re-training from scratch. We categorize research works systemically and provide in-depth comparisons and discussion. We also discuss existing challenges and highlight future directions to facilitate research in this field. We release the paper list at https://github.com/hyintell/awesome-refreshing-llms

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 153 of its 196 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

153 of 196 references.

References · 153

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  11. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  12. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  8. 2019Language Models as Knowledge Bases?cited by 1,936
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  13. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  14. 2023PaLM 2 Technical Reportcited by 1,162
  15. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  16. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  17. 2016HyperNetworkscited by 1,027
  18. 2022Mass-Editing Memory in a Transformercited by 1,001
  19. 2019How Can We Know What Language Models Know?cited by 959
  20. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  21. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  22. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  23. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  24. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  25. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  26. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  27. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  28. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  29. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  30. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  31. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  32. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  33. 2021Editing Factual Knowledge in Language Modelscited by 553
  34. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  35. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  36. 2023Active Retrieval Augmented Generationcited by 516
  37. 2021Fast Model Editing at Scalecited by 498
  38. 2022Memory-Based Model Editing at Scalecited by 447

50 most cited of 153 All 153 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 15; 2026: 3.

33 papers.

Sort:

By area:

  1. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  2. 2024Continual Learning for Large Language Models: A Surveycited by 147
  3. 2024Large Multimodal Agents: A Surveycited by 105
  4. 2024SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuningcited by 32
  5. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  6. 2024Knowledge Editing in Language Models via Adapted Direct Preference Optimizationcited by 17
  7. 2023GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Modelscited by 16
  8. 2024Learning to Poison Large Language Models for Downstream Manipulationcited by 16
  9. 2024Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformerscited by 15
  10. 2024Towards LifeSpan Cognitive Systemscited by 13
  11. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 12
  12. 2024Model Editing for LLMs4Code: How Far are We?cited by 10
21 more
  1. 2025LLMs as Repositories of Factual Knowledge: Limitations and Solutionscited by 10
  2. 2024ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domainscited by 8
  3. 2025Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agentscited by 6
  4. 2025Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradationcited by 6
  5. 2025Memorizing is Not Enough: Deep Knowledge Injection Through Reasoningcited by 4
  6. 2025Provable Benefits of In-Tool Learning for Large Language Modelscited by 4
  7. 2026STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generationcited by 4
  8. 2024Enhancing Conflict Resolution in Language Models via Abstract Argumentationcited by 3
  9. 2025KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearningcited by 3
  10. 2024Adversarial Attacks on Data Attributioncited by 2
  11. 2025Unlocking Pretrained LLMs for Motion-Related Multimodal Generation: A Fine-Tuning Approach to Unify Diffusion and Next-Token Predictioncited by 2
  12. 2026Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decodingcited by 2
  13. 2025Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generationcited by 1
  14. 2025SAGE: Exploring the Boundaries of Unsafe Concept Domain with Semantic-Augment Erasingcited by 1
  15. 2025Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systemscited by 1
  16. 2025SDA-PLANNER: State-Dependency Aware Adaptive Planner for Embodied Task Planningcited by 1
  17. 2023Sentiment Analysis in Digital Spaces: An Overview of Reviewscited by 0
  18. 2025ICA-RAG: Information Completeness Guided Adaptive Retrieval-Augmented Generation for Disease Diagnosiscited by 0
  19. 2025From Retrieval to Generation: Unifying External and Parametric Knowledge for Medical Question Answeringcited by 0
  20. 2025AIAuditTrack: A Framework for AI Security systemcited by 0
  21. 2026CLEAR: Cross-Source Evidence Adjudication for Large Language Models in Medicinecited by 0