Self-Generated Error Training for Token Editing in Diffusion Language Models
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
Token-to-token (T2T) editing lets LLaDA2.1 revise committed tokens during block-diffusion decoding. The released recipe trains this editor on random vocabulary corruptions, but at inference the editor sees the model's own fluent, high-confidence draft errors instead. We study this training-inference mismatch and propose self-generated T2T, which performs a no-gradient draft pass, fills masked positions with predicted tokens, and supervises recovery in a second pass under these self-generated corruptions. We implement the update as a short LoRA continued-pretraining pass on LLaDA2.1-mini and evaluate on several benchmarks under the official Q-Mode T2T procedure with unchanged inference parameters. The method generally improves accuracy while reducing T2T edit intensity, mitigating failure modes such as final-digit transcription errors after otherwise correct reasoning and excessive self-correction before short factual answers.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 20 of its 22 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
20 of 22 references.
Closest ancestors · 13
Same direction, newest first
- 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models20 Apr
- 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models12 Feb
- 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
- 2026BackPlay: Head-Only Look-Back Self-Correction for Diffusion Language Models10 Jan
- 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
- 2025Dream 7B: Diffusion Large Language Models21 Aug
- 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
- 2025Large Language Diffusion Models14 Feb
- 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
- 2024Simple and Effective Masked Diffusion Language Models11 Jun
- 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
- 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
Other references · 7
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
- 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
- 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
- 2023CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?cited by 90
- 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 43
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
- 2026Edit-Based Refinement for Parallel Masked Diffusion Language Models
- 2026Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
- 2026Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models
- 2026DEdit: Iterative Draft Editing for Speculative Decoding
- 2026Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
- 2026Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models
- 2025Fine-Tuning Masked Diffusion for Provable Self-Correction