mlsanityData to 14 Oct 2026

In-Place Instruction Following in Diffusion Language Models

arXiv 2609.07160 ↗First posted 7 Sep 2026cs.CL cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Large Language Models (dLLMs) generate text via bidirectional iterative denoising, naturally supporting user-specified constraints anchored at arbitrary output positions, a paradigm known as In-place Prompting (IPP). We formalize this as the In-place Instruction Following (IIF) task and construct IIF-Bench, a hierarchical benchmark spanning literal, style, and discourse-function constraints, paired with a rubric-based local-global evaluation protocol. An inference-time attention-bias probe suggests that vanilla dLLMs often under-prioritize constraint spans during denoising. We then propose GRAFT, an IPP-oriented post-training framework combining constraint-aware SFT and preference optimization. On four representative dLLMs, GRAFT raises the average IIF score from 57.75 to 73.10 (+15.35 points), with absolute gains of 15.91 and 15.57 points on literal and discourse-function constraints, while preserving general generation ability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 30 of its 52 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

30 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  2. 2025Unlocking the Potential of Diffusion Language Models through Template Infilling13 Oct
  3. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  4. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  5. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  6. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  4. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  5. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  6. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  7. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  8. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
  9. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  10. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 101
  11. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  12. 2024"We Need Structured Output": Towards User-centered Constraints on Large Language Model Outputcited by 46
12 more
  1. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  2. 2025CRANE: Reasoning with constrained LLM generationcited by 38
  3. 2025The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMscited by 34
  4. 2024Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?cited by 30
  5. 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 26
  6. 2024Style-Specific Neurons for Steering LLMs in Text Style Transfercited by 24
  7. 2025Thinking Inside the Mask: In-Place Prompting in Diffusion LLMscited by 13
  8. 2024CAT-LLM: Style-enhanced Large Language Models with Text Style Definition for Chinese Article-style Transfercited by 10
  9. 2022AutoTemplate: A Simple Recipe for Lexically Constrained Text Generationcited by 6
  10. 2023Prefix-Tuning Based Unsupervised Text Style Transfercited by 6
  11. 2024LMStyle Benchmark: Evaluating Text Style Transfer for Chatbotscited by 5
  12. 2024Empowering Character-level Text Infilling by Eliminating Sub-Tokenscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unlocking Prompt Infilling Capability for Diffusion Language Models
  2. 2026Diffusion Language Models: An Experimental Analysis
  3. 2026Reducing Pretraining-Generation Mismatch in Diffusion Language Models
  4. 2026Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
  5. 2025Context-Aware Initialization for Reducing Generative Path Length in Diffusion Language Models
  6. 2026DIP: Dynamic In-Context Planner For Diffusion Language Models
  7. 2026Diffusion Language Models Are Natively Length-Aware
  8. 2026Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants