mlsanityData to 14 Oct 2026

DeepEdit: Knowledge Editing as Decoding with Constraints

arXiv 2401.10471 ↗First posted 19 Jan 2024cs.CL cs.AIcited by 33

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

How to edit the knowledge in multi-step reasoning has become the major challenge in the knowledge editing (KE) of large language models (LLMs). The difficulty arises because the hallucinations of LLMs during multi-step reasoning often lead to incorrect use of new knowledge and incorrect answers. To address this issue, we design decoding constraints to "regulate" LLMs' reasoning, enhancing logical coherence when incorporating new knowledge. We propose a new KE framework: DEEPEDIT (Depth-first Search-based Constrained Decoding for Knowledge Editing), which enhances LLMs's ability to generate coherent reasoning chains with new knowledge through depth-first search. Our search selects the most important knowledge that satisfies our constraints as the reasoning step to efficiently increase the reasoning depth. In addition to DEEPEDIT, we propose two new KE benchmarks: MQUAKE-2002 and MQUAKE-HARD, which provide more precise and challenging assessments of KE approaches. Qualitatively, DEEPEDIT enables LLMs to produce succinct and coherent reasoning chains involving new knowledge. Quantitatively, it yields significant improvements on multiple KE benchmarks.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 43 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

32 of 43 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  7. 2022Mass-Editing Memory in a Transformercited by 1,001
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  9. 2021Editing Factual Knowledge in Language Modelscited by 553
  10. 2021Fast Model Editing at Scalecited by 498
  11. 2022Memory-Based Model Editing at Scalecited by 447
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
20 more
  1. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  2. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  3. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  4. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  5. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  6. 2022Generating Sequences by Learning to Self-Correctcited by 277
  7. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  8. 2020Modifying Memories in Transformer Modelscited by 224
  9. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  10. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  11. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  12. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 149
  13. 2021Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefscited by 79
  14. 2020Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoningcited by 65
  15. 2023Tractable Control for Autoregressive Language Generationcited by 65
  16. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  17. 2022Controllable Text Generation with Neurally-Decomposed Oraclecited by 41
  18. 2023The ART of LLM Refinement: Ask, Refine, and Trustcited by 29
  19. 2023BOLT: Fast Energy-based Controlled Text Generation with Tunable Biasescited by 24
  20. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 11; 2026: 6.

33 papers.

Sort:

By area:

  1. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  2. 2024Multi-hop Question Answering under Temporal Knowledge Editingcited by 36
  3. 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
  4. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 34
  5. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  6. 2024Can Editing LLMs Inject Harm?cited by 22
  7. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  8. 2024Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulnesscited by 17
  9. 2024Adaptive Token Biaser: Knowledge Editing via Biasing Key Entitiescited by 14
  10. 2024Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Modelscited by 12
  11. 2025Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editingcited by 12
  12. 2024Neighboring Perturbations of Knowledge Editing on Large Language Modelscited by 10
21 more
  1. 2024Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understandingcited by 9
  2. 2024StruEdit: Structured Outputs Enable the Fast and Accurate Knowledge Editing for Large Language Modelscited by 9
  3. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  4. 2025Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editingcited by 8
  5. 2025Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Callingcited by 7
  6. 2024RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learningcited by 5
  7. 2025K-Edit: Language Model Editing with Contextual Knowledge Awarenesscited by 5
  8. 2025Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QAcited by 4
  9. 2024CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMscited by 3
  10. 2026PromptCD: Test-Time Behavior Enhancement via Polarity-Prompt Contrastive Decodingcited by 3
  11. 2024Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledgecited by 1
  12. 2024PFME: A Modular Approach for Fine-grained Hallucination Detection and Editing of Large Language Modelscited by 1
  13. 2025EtCon: Edit-then-Consolidate for Reliable Knowledge Editingcited by 1
  14. 2026Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editingcited by 1
  15. 2026Rule-Compliant Visual Spatial Planning for Multimodal Large Language Modelscited by 1
  16. 2025ADS-Edit: A Multimodal Knowledge Editing Dataset for Autonomous Driving Systemscited by 0
  17. 2025Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decompositioncited by 0
  18. 2025Stemming Hallucination in Language Models Using a Licensing Oraclecited by 0
  19. 2026Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Storiescited by 0
  20. 2026Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMscited by 0
  21. 2026ArtAnno: Annotating Implicit Semantics in Artworks through LLM Agent-Driven Bidirectional Human-AI Augmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models
  2. 2026CRANE: Knowledge Editing for Reasoning MLLMs
  3. 2025Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
  4. 2024Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing
  5. 2026MoEEdit: Efficient and Routing-Stable Knowledge Editing for Mixture-of-Experts LLMs
  6. 2026Learning to Edit Knowledge via Instruction-based Chain-of-Thought Prompting
  7. 2024Knowledge Editing through Chain-of-Thought
  8. 2024ConKE: Conceptualization-Augmented Knowledge Editing in Large Language Models for Commonsense Reasoning