mlsanityData to 14 Oct 2026

Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models

arXiv 2508.03741 ↗First posted 1 Aug 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) often retain inaccurate or outdated information from pre-training, leading to incorrect predictions or biased outputs during inference. While existing model editing methods can address this challenge, they struggle with editing large amounts of factual information simultaneously and may compromise the general capabilities of the models. In this paper, our empirical study demonstrates that it is feasible to edit the internal representations of LLMs and replace the entities in a manner similar to editing natural language inputs. Based on this insight, we introduce the Latent Knowledge Scalpel (LKS), an LLM editor that manipulates the latent knowledge of specific entities via a lightweight hypernetwork to enable precise and large-scale editing. Experiments conducted on Llama-2 and Mistral show even with the number of simultaneous edits reaching 10,000, LKS effectively performs knowledge editing while preserving the general abilities of the edited LLMs. Code is available at: https://github.com/Linuxin-xxx/LKS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 49 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

43 of 49 references.

References · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2023Mistral 7Bcited by 3,164
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2019Language Models as Knowledge Bases?cited by 1,936
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  10. 2022Mass-Editing Memory in a Transformercited by 1,001
  11. 2019How Can We Know What Language Models Know?cited by 959
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
31 more
  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  2. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  3. 2021Editing Factual Knowledge in Language Modelscited by 553
  4. 2021Fast Model Editing at Scalecited by 498
  5. 2022Memory-Based Model Editing at Scalecited by 447
  6. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  7. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  8. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  9. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  10. 2020Autoregressive Entity Retrievalcited by 334
  11. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  12. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  13. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  14. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  15. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
  16. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  17. 2021Implicit Representations of Meaning in Neural Language Modelscited by 180
  18. 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 173
  19. 2021Meta-learning via Language Model In-context Tuningcited by 164
  20. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141
  21. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 116
  22. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  23. 2024Large Language Models Meet NLP: A Surveycited by 103
  24. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
  25. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 92
  26. 2023Massive Editing for Large Language Models via Meta Learningcited by 63
  27. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  28. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  29. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  30. 2022Position-based Prompting for Health Outcome Generationcited by 7
  31. 2023Commonsense Knowledge Transfer for Pre-trained Language Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Models
  2. 2024Uncovering Overfitting in Large Language Model Editing
  3. 2024Learning to Edit: Aligning LLMs with Knowledge Editing
  4. 2025Benchmarking and Rethinking Knowledge Editing for Large Language Models
  5. 2025Massive Editing for Large Language Models Based on Dynamic Weight Generation
  6. 2025Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing
  7. 2025Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoning
  8. 2024Knowledge Graph Enhanced Large Language Model Editing