mlsanityData to 14 Oct 2026

Can We Edit Factual Knowledge by In-Context Learning?

arXiv 2305.12740 ↗First posted 22 May 2023cs.CLcited by 285

Not on the map. Why?

Cited by 26 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/Zce1112zslx/IKE.

Key facts. Cited by 285 papers in the map (data through 14 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

32 of 45 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  11. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  12. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
20 more
  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  5. 2022Mass-Editing Memory in a Transformercited by 1,001
  6. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  7. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  9. 2021Editing Factual Knowledge in Language Modelscited by 553
  10. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  11. 2021Fast Model Editing at Scalecited by 498
  12. 2022Memory-Based Model Editing at Scalecited by 447
  13. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  14. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  15. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
  16. 2022Prompting GPT-3 To Be Reliablecited by 260
  17. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 256
  18. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  19. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  20. 2023Language Models are Few-shot Learners for Prognostic Predictioncited by 37

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 20; 2024: 110; 2025: 102; 2026: 53.

285 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  2. 2024Rethinking Machine Unlearning for Large Language Modelscited by 257
  3. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  4. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  5. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  6. 2023PMET: Precise Model Editing in a Transformercited by 173
  7. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  8. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  9. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  10. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  11. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
  12. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 93
38 more
  1. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  2. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 86
  3. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
  4. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  5. 2024Factuality of Large Language Models: A Surveycited by 66
  6. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  7. 2024Larimar: Large Language Models with Episodic Memory Controlcited by 53
  8. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  9. 2024Distinguishing the Knowable from the Unknowable with Language Modelscited by 47
  10. 2023Can We Edit Multimodal Large Language Models?cited by 44
  11. 2024Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learningcited by 40
  12. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  13. 2024Uncovering Overfitting in Large Language Model Editingcited by 39
  14. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 36
  15. 2024Multi-hop Question Answering under Temporal Knowledge Editingcited by 36
  16. 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
  17. 2023Evaluating the Reversal Curse in Model Editingcited by 34
  18. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 34
  19. 2024Conformity in Large Language Modelscited by 34
  20. 2025Reinforced Lifelong Editing for Language Modelscited by 34
  21. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  22. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  23. 2024VLKEB: A Large Vision-Language Model Knowledge Editing Benchmarkcited by 30
  24. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  25. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  26. 2024Studying Large Language Model Behaviors Under Context-Memory Conflicts With Real Documentscited by 27
  27. 2025The Mirage of Model Editing: Revisiting Evaluation in the Wildcited by 27
  28. 2023DocTER: Evaluating Document-based Knowledge Editingcited by 26
  29. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 26
  30. 2024MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editingcited by 25
  31. 2024DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMscited by 24
  32. 2023PokeMQA: Programmable knowledge editing for Multi-hop Question Answeringcited by 23
  33. 2024Long-form evaluation of model editingcited by 23
  34. 2025MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledgecited by 23
  35. 2023Editing Personality for Large Language Modelscited by 22
  36. 2024MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistencycited by 22
  37. 2024Can Editing LLMs Inject Harm?cited by 22
  38. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21

50 most cited of 285 All 285 (JSON)