mlsanityData to 14 Oct 2026

DocTER: Evaluating Document-based Knowledge Editing

arXiv 2308.09954 ↗First posted 19 Aug 2023cs.CL cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Knowledge editing aims to correct outdated or inaccurate knowledge in neural networks. In this paper, we explore knowledge editing using easily accessible documents instead of manually labeled factual triples employed in earlier research. To advance this field, we establish the first evaluation benchmark, textitDocTER, featuring Documents containing counterfactual knowledge for editing. A comprehensive four-perspective evaluation is introduced: Edit Success, Locality, Reasoning, and Cross-lingual Transfer. To adapt conventional triplet-based knowledge editing methods for this task, we develop an Extract-then-Edit pipeline that extracts triples from documents before applying existing methods. Experiments on popular knowledge editing methods demonstrate that editing with documents presents significantly greater challenges than using triples. In document-based scenarios, even the best-performing in-context editing approach still lags behind by 10 points in editing success when compared to using gold triples. This observation also holds for both reasoning and cross-lingual test sets. We further analyze key factors influencing task performance, including the quality of extracted triples, the frequency and position of edited knowledge in documents, various methods for enhancing reasoning, and performance differences across various directions in cross-lingual knowledge editing, which provide valuable insights for future research.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 19 Aug 2023.

Builds on

34 of 56 references.

References · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  9. 2019Language Models as Knowledge Bases?cited by 1,936
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  11. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  12. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
22 more
  1. 2022Mass-Editing Memory in a Transformercited by 1,001
  2. 2019How Can We Know What Language Models Know?cited by 959
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  4. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  5. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  6. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  7. 2021Editing Factual Knowledge in Language Modelscited by 553
  8. 2021Fast Model Editing at Scalecited by 498
  9. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  10. 2022Memory-Based Model Editing at Scalecited by 447
  11. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  12. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  13. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  14. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  15. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  16. 2020Modifying Memories in Transformer Modelscited by 224
  17. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  18. 2023PMET: Precise Model Editing in a Transformercited by 173
  19. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92
  20. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 67
  21. 2022Bidirectional Language Models Are Also Few-shot Learnerscited by 48
  22. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 14; 2025: 6; 2026: 1.

26 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  4. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  5. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  6. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  7. 2024Determinants of LLM-assisted Decision-Makingcited by 56
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 51
  9. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 36
  10. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  11. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  12. 2024Perturbation-Restrained Sequential Model Editingcited by 28
14 more
  1. 2025MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledgecited by 23
  2. 2023Editing Personality for Large Language Modelscited by 22
  3. 2024RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuningcited by 21
  4. 2024Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Modelscited by 20
  5. 2024Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Modelscited by 19
  6. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  7. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  8. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  9. 2023Cross-Lingual Knowledge Editing in Large Language Modelscited by 11
  10. 2024Event-level Knowledge Editingcited by 9
  11. 2025Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editingcited by 8
  12. 2024Fine-Grained Self-Endorsement Improves Factuality and Reasoningcited by 7
  13. 2025Large Language Models as Computable Approximations to Solomonoff Inductioncited by 6
  14. 2026Benchmarking Knowledge Editing using Logical Rulescited by 0