mlsanityData to 14 Oct 2026

Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansion

arXiv 2502.12598 ↗First posted 18 Feb 2025cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Adapting large language models (LLMs) to new and diverse knowledge is essential for their lasting effectiveness in real-world applications. This survey provides an overview of state-of-the-art methods for expanding the knowledge of LLMs, focusing on integrating various knowledge types, including factual information, domain expertise, language proficiency, and user preferences. We explore techniques, such as continual learning, model editing, and retrieval-based explicit adaptation, while discussing challenges like knowledge consistency and scalability. Designed as a guide for researchers and practitioners, this survey sheds light on opportunities for advancing LLMs as adaptable and robust knowledge systems.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 98 of its 129 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

98 of 129 references.

References · 98

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  9. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  11. 2022Mass-Editing Memory in a Transformercited by 1,001
  12. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
38 more
  1. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  2. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  3. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  4. 2021Editing Factual Knowledge in Language Modelscited by 553
  5. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  6. 2022Memory-Based Model Editing at Scalecited by 447
  7. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  8. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  9. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  10. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  11. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  12. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  13. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  14. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  15. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  16. 2022Prompting GPT-3 To Be Reliablecited by 260
  17. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 250
  18. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  19. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 231
  20. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
  21. 2020Modifying Memories in Transformer Modelscited by 224
  22. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  23. 2023Continual Pre-training of Language Modelscited by 198
  24. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  25. 2023Progressive Prompts: Continual Learning for Language Modelscited by 197
  26. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  27. 2023PMET: Precise Model Editing in a Transformercited by 173
  28. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 166
  29. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  30. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 158
  31. 2024CodeGemma: Open Code Models Based on Gemmacited by 150
  32. 2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
  33. 2024Continual Learning for Large Language Models: A Surveycited by 147
  34. 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
  35. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  36. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 126
  37. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  38. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 115

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Is Extending Modality The Right Path Towards Omni-Modality?cited by 5
  2. 2026RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real Worldcited by 4
  3. 2025QBD-RankedDataGen: Generating Custom Ranked Datasets for Improving Query-By-Document Search Using LLM-Reranking with Reduced Human Effortcited by 0
  4. 2026The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editingcited by 0
  5. 2026LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAGcited by 0
  6. 2026Greener Than Humans? Environmental Attitudes in Large Language Modelscited by 0
  7. 2026AI Alignment through a Game-theoretic Lens: A Surveycited by 0
  8. 2026CLEAR: Cross-Source Evidence Adjudication for Large Language Models in Medicinecited by 0