mlsanityData to 14 Oct 2026

Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models

arXiv 2606.10829 ↗First posted 9 Jun 2026cs.CL cs.AIcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration, but this parallelism is fragile: positions that are individually confident may be unsafe to commit together when their predictions are coupled. Existing training-free samplers such as Top-(k), Fast-dLLM, and EB-Sampler mainly control how many tokens to reveal, while often ranking candidates by token-wise scores that ignore interactions within the selected set. We propose ADAS, a training-free reranking rule that leaves the base sampler's stopping rule unchanged and greedily discounts each token-wise confidence score according to its attention to already selected positions, weighted by their prediction uncertainty. Across LLaDA-8B-Base and Dream-7B-Base on the reasoning benchmarks GSM8K and MATH500 and the code benchmarks HumanEval and MBPP, plugging ADAS into all three samplers improves low-NFE performance at matched denoiser evaluations by (9.11) and (10.46) percentage points on average, respectively, with (3.1%) per-forward runtime overhead. Code is available at https://github.com/yusufsahin99/ADAS.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 22 of its 22 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

22 of 22 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models10 Feb
  2. 2025Learning Unmasking Policies for Diffusion Language Models9 Dec
  3. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  4. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models4 Nov
  5. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing24 Oct
  6. 2025Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies7 Oct
  7. 2025Dream 7B: Diffusion Large Language Models21 Aug
  8. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models23 Jun
  9. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  10. 2025Large Language Diffusion Models14 Feb
  11. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
1 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun

Other references · 9

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  6. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  7. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decodingcited by 27
  8. 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMscited by 14
  9. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inferencecited by 0
  2. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reportedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
  2. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  3. 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
  4. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  5. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  6. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Models
  7. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  8. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models