mlsanityData to 14 Oct 2026

Knowledge Editing for Masked Diffusion Language Models

arXiv 2606.03924 ↗First posted 2 Jun 2026cs.CLcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Knowledge editing aims to update or correct factual knowledge in a language model. A widely used approach, locate-then-edit, first localizes a fact within the model and then edits the weights there. To date, such methods have been developed exclusively for autoregressive models (ARMs). Whether they work for masked diffusion models (MDMs), which model text bidirectionally and generate by iterative denoising rather than next-token prediction, remains an open question. We address it by transferring locate-then-edit to MDMs and comparing multiple MDMs with their matched ARMs. Our central finding has two parts. First, where an edit should be applied transfers between them: the same early-to-mid-layer MLP at the last subject token is most effective for both. Second, this shared location does not guarantee a shared outcome. Single-token edits succeed in both, but as targets grow longer, editing degrades far more sharply in the MDMs than in the ARMs. The failure stems from how the edited fact is generated: producing a multi-token target passes through intermediate states in which the target is partially unmasked, for which the edit was never optimized. Guided by this diagnosis, we introduce a simple correction that optimizes the edit including such states, substantially restoring multi-token performance. Our code is available at https://github.com/holi-lab/MDM-KE.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 24 of its 37 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

24 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoders5 Feb
  2. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs31 Aug
  3. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct

Other references · 21

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2022Mass-Editing Memory in a Transformercited by 1,001
  5. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  6. 2021Editing Factual Knowledge in Language Modelscited by 553
  7. 2022Memory-Based Model Editing at Scalecited by 447
  8. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  9. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  10. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  11. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  12. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
9 more
  1. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  2. 2020Modifying Memories in Transformer Modelscited by 224
  3. 2021Towards Continual Knowledge Learning of Language Modelscited by 174
  4. 2023PMET: Precise Model Editing in a Transformercited by 173
  5. 2024A Unified Framework for Model Editingcited by 57
  6. 2025Scaling Laws for Forgetting during Finetuning with Pretraining Data Injectioncited by 35
  7. 2025Context-Robust Knowledge Editing for Language Modelscited by 8
  8. 2025Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMscited by 2
  9. 2025SUIT: Knowledge Editing with Subspace-Aware Key-Value Mappingscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What does FFN compression change downstream? Same-state causal restoration in diffusion language modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TimeROME-DLM: Temporal Causal Tracing and Low-Rank Inference-Time Knowledge Editing for Masked Diffusion Language Models
  2. 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
  3. 2026Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
  4. 2024Scaling up Masked Diffusion Models on Text
  5. 2026On Trajectory-Aware Training for Masked Diffusion Language Models
  6. 2026Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
  7. 2026Reasoning with Latent Tokens in Diffusion Language Models
  8. 2025Rethinking Residual Distribution in Locate-then-Edit Model Editing