mlsanityData to 7 Oct 2026

DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs

arXiv 2603.12996 ↗First posted 13 Mar 2026cs.LGcited by 13

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Parallel decoding for Diffusion LLMs (dLLMs) is difficult because each denoising step provides only token-wise marginal distributions, while unmasking multiple tokens simultaneously requires accounting for inter-token dependencies. We propose Dependency-Aware Parallel Decoding (DAPD), a simple, training-free decoding method that uses self-attention to induce a conditional dependency graph over masked tokens. At each iteration, edges in this graph capture strong token interactions, while non-edges indicate weak dependence. Parallel decoding is then reduced to selecting an independent set on the graph and unmasking the selected tokens in parallel. This avoids co-updating strongly coupled tokens without auxiliary models or retraining. Experiments on LLaDA and Dream show that DAPD improves the accuracy-steps trade-off over existing methods and enables more globally distributed parallel updates that better exploit the any-order generation capability of dLLMs. The project is available at https://ai-isl.github.io/dapd

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 37 of its 44 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

37 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMs6 Feb
  2. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs6 Oct
  3. 2025dParallel: Learnable Parallel Decoding for dLLMs30 Sep
  4. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding29 Sep
  5. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding31 May
  6. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May

Other references · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
19 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  2. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  3. 2025Large Language Diffusion Modelscited by 872
  4. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 712
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  7. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  8. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  9. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  10. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  12. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  13. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 208
  14. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  15. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmaskingcited by 128
  16. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 44
  17. 2025Ideas in Inference-time Scaling can Benefit Generative Pre-training Algorithmscited by 12
  18. 2026Breaking the Factorization Barrier in Diffusion Language Modelscited by 11
  19. 2025Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Modelscited by 2
  2. 2026Attention-Discounted Adaptive Sampler for Masked Diffusion Language Modelscited by 2
  3. 2026The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Modelscited by 1
  4. 2026Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Modelscited by 1
  5. 2026EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Modelscited by 1
  6. 2026Supportive Token Revealing for Fast Diffusion Language Model Decodingcited by 1
  7. 2026Mean-Field Parallel Decoding for Discrete Diffusion Language Modelscited by 0
  8. 2026CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inferencecited by 0
  9. 2026Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inferencecited by 0
  10. 2026DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Modelscited by 0
  11. 2026FastGuide: Accelerating Reward Guidance for Diffusion Large Language Modelscited by 0
  12. 2026Reliable Parallel Decoding in Masked Diffusion Language Modelscited by 0
1 more
  1. 2026Hierarchical Continuous Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026DMax: Aggressive Parallel Decoding for dLLMs
  2. 2026Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models
  3. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
  4. 2026PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
  5. 2026Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
  6. 2025CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
  7. 2026S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation
  8. 2026DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding