mlsanityData to 7 Oct 2026

When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models

arXiv 2606.21802 ↗First posted 19 Jun 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Standard tokenwise diffusion LMs keep training corruption and inference commitment at token granularity throughout denoising. At high noise, this leaves scattered local fragments rather than coherent evidence, making it hard to form early coarse structure, exactly what planning-sensitive generation requires. Hierarchical planning methods add coarse stages to separate planning from wording, but they need extra planners, block latents, or two stage designs. We propose Noise Dependent Granularity Control (NDGC), a single-level diffusion method that uses the noise level as a granularity cue. NDGC aligns training exposure and inference commitment with denoising progress. High noise steps use coherent token groups to support early meaning commitment, while low noise steps return to token level refinement. This creates planning like coarse to fine denoising without an explicit planner or hierarchical architecture. Across controlled tests, ablations, and WritingPrompts, NDGC shows earlier skeleton formation, better ordered recovery, and healthier outputs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 47 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

27 of 47 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Reward-Weighted Sampling: Enhancing Non-Autoregressive Characteristics in Masked Diffusion LLMs31 Aug
  2. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusion23 Apr
  3. 2025Generalized Interpolating Discrete Diffusion6 Mar
  4. 2024Conditional [MASK] Discrete Diffusion Language Model10 Nov
  5. 2024Scaling up Masked Diffusion Models on Text24 Oct
  6. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  7. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  9. 2023DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models9 Oct
  10. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation6 May
  11. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  12. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov

Other references · 15

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 232
  6. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  7. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  8. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  9. 2019Strategies for Structuring Story Generationcited by 124
  10. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  11. 2020Content Planning for Neural Story Generation with Aristotelian Rescoringcited by 92
  12. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
3 more
  1. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  2. 2024Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancementcited by 30
  3. 2024Ex3: Automatic Novel Writing by Extracting, Excelsior and Expandingcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning
  2. 2025Planned Diffusion
  3. 2026Support Before Frequency in Discrete Diffusion
  4. 2026Reducing Pretraining-Generation Mismatch in Diffusion Language Models
  5. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  6. 2026Measuring Temporal Linguistic Emergence in Diffusion Language Models
  7. 2026On Trajectory-Aware Training for Masked Diffusion Language Models
  8. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models