mlsanityData to 14 Oct 2026

Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models

arXiv 2604.02560 ↗First posted 2 Apr 2026cs.CLcited by 8

On the map: Diffusion models → Discrete and text diffusion

also in 1 more
Abstract (arXiv)

Discrete diffusion language models (dLLMs) accelerate text generation by unmasking multiple tokens in parallel. However, parallel decoding introduces a distributional mismatch: it approximates the joint conditional using a fully factorized product of per-token marginals, which degrades output quality when selected tokens are strongly dependent. We propose DEMASK (DEpendency-guided unMASKing), a lightweight dependency predictor that attaches to the final hidden states of a dLLM. In a single forward pass, it estimates pairwise conditional influences between masked positions. Using these predictions, a greedy selection algorithm identifies positions with bounded cumulative dependency for simultaneous unmasking. Under a sub-additivity assumption, we prove this bounds the total variation distance between our parallel sampling and the model's joint. Empirically, DEMASK achieves 1.7-2.2× speedup on Dream-7B while matching or improving accuracy compared to confidence-based and KL-based baselines.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 26 of its 31 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

26 of 31 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  2. 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
  3. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  4. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing24 Oct
  5. 2025Dream 7B: Diffusion Large Language Models21 Aug
  6. 2025Large Language Diffusion Models14 Feb
  7. 2024Discrete Copula Diffusion2 Oct
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  9. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  11. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  12. 2025Kimi K2: Open Agentic Intelligencecited by 567
3 more
  1. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  2. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  3. 2025Enabling Approximate Joint Sampling in Diffusion LMscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Accelerating Discrete Diffusion Models with Parallel-In-Time Samplingcited by 4
  2. 2026Limits of Confidence in Diffusioncited by 2
  3. 2026Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Modelscited by 1
  4. 2026Mean-Field Parallel Decoding for Discrete Diffusion Language Modelscited by 0
  5. 2026Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMscited by 0
  6. 2026Unmask the State: When Does State Adaptation Matter for Masked Diffusion Language Modelscited by 0
  7. 2026Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoningcited by 0
  8. 2026Hierarchical Continuous Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMs
  2. 2026DOS: Dependency-Oriented Sampler for Masked Diffusion Language Models
  3. 2026Locally Coherent Parallel Decoding in Diffusion Language Models
  4. 2026Reliable Parallel Decoding in Masked Diffusion Language Models
  5. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
  6. 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs
  7. 2026DMax: Aggressive Parallel Decoding for dLLMs
  8. 2025Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models