mlsanityData to 7 Oct 2026

Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models

arXiv 2602.19619 ↗First posted 23 Feb 2026cs.LGcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete diffusion language models (dLLMs) provide a fast and flexible alternative to autoregressive models (ARMs) via iterative denoising with parallel updates. However, their evaluation is challenging: existing metrics conflate denoiser approximation error with sampler-induced error from the sampling dynamics, a problem that does not arise for ARMs whose autoregressive sampling exactly reflects the learned probability model. We introduce a sampler-centric oracle framework that replaces learned denoisers with an exact Hidden Markov Model posterior derived from a ground-truth Markov chain, isolating sampler-induced error in a controlled setting. We show that few-step discrete diffusion samplers are not distributionally correct even under an oracle denoiser, with transition-level mismatch that vanishes only as the number of steps approaches the sequence length. Moreover, improvements in negative log-likelihood (NLL), generative perplexity (GenPPL), or MAUVE do not imply correct sampling. Code is available at https://luhantang.github.io/dllm_sampler

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 26 of its 42 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

26 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  4. 2025Large Language Diffusion Models14 Feb
  5. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  6. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  7. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  7. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  9. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  10. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  11. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  12. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
6 more
  1. 2023Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspectivecited by 26
  2. 2025Discrete Diffusion Models: Novel Analysis and New Sampler Guaranteescited by 20
  3. 2025Enabling Approximate Joint Sampling in Diffusion LMscited by 9
  4. 2025RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidancecited by 7
  5. 2026Dynamical Regimes of Multimodal Diffusion Modelscited by 4
  6. 2026dVoting: Fast Voting for dLLMscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reportedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Consistent Diffusion Language Models
  2. 2026The Diffusion Duality, Chapter II: Psi-Samplers
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
  4. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement
  5. 2026Discrete Diffusion Models: A Unified Framework from Tokenization to Generation
  6. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
  7. 2026Optimizers for Diffusion Models: A Controlled Benchmark
  8. 2026Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models