mlsanityData to 7 Oct 2026

The Diffusion Duality, Chapter II: Psi-Samplers

arXiv 2602.21185 ↗First posted 24 Feb 2026cs.LGcited by 12

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Uniform-state discrete diffusion models excel at few-step generation and guidance due to their ability to self-correct, making them preferred over autoregressive or Masked diffusion models in these settings. However, their sampling quality plateaus with ancestral samplers as the number of steps increases. We introduce a family of Predictor-Corrector (PC) samplers for discrete diffusion that generalize prior methods and apply to arbitrary noise processes. When paired with uniform-state diffusion, our samplers outperform ancestral sampling on both language and image modeling, achieving lower generative perplexity at matched unigram entropy on OpenWebText and better FID/IS scores on CIFAR10. Crucially, unlike conventional samplers, our PC methods continue to improve with more sampling steps. Taken together, these findings call into question the assumption that Masked diffusion is the inevitable future of diffusion-based language modeling. Beyond sampling, we develop a memory-efficient curriculum for the Gaussian relaxation training phase, reducing training time by 25% and memory by 33% compared to Duo while maintaining comparable perplexity on OpenWebText and LM1B and strong downstream performance. We release code, checkpoints, and a video-tutorial on: https://s-sahoo.com/duo-ch2

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 58 of its 74 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

58 of 74 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026Scaling Beyond Masked Diffusion Language Models16 Feb
  2. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  3. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  4. 2025The Diffusion Duality12 Jun
  5. 2025Generalized Interpolating Discrete Diffusion6 Mar
  6. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  7. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  8. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  9. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  10. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  11. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  12. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
3 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  2. 2022Continuous diffusion for categorical data28 Nov
  3. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov

Other references · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
31 more
  1. 2016Pointer Sentinel Mixture Modelscited by 2,992
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  8. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  9. 2021Variational Diffusion Modelscited by 1,145
  10. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  11. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  12. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  13. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  14. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  15. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  16. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  17. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  18. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  19. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  20. 2023Protein Design with Guided Discrete Diffusioncited by 154
  21. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  22. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  23. 2024Generator Matching: Generative modeling with arbitrary Markov processescited by 82
  24. 2023Generative Diffusion Models on Graphs: Methods and Applicationscited by 79
  25. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 54
  26. 2025GenMol: A Drug Discovery Generalist with Discrete Diffusioncited by 47
  27. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  28. 2024textitJump Your Steps: Optimizing Sampling Schedule of Discrete Diffusion Modelscited by 40
  29. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  30. 2025Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokenscited by 7
  31. 2025Partition Generative Modeling: Masked Modeling Without Maskscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026ELF: Embedded Language Flowscited by 27
  2. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Spacecited by 8
  3. 2026Simplex Relaxation for Discrete Diffusioncited by 2
  4. 2026From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Modelscited by 1
  5. 2026From Truncation to Commitment: Persistent Context in Uniform Discrete Diffusioncited by 1
  6. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusioncited by 1
  7. 2026Diffusion Language Models for Speech Recognitioncited by 0
  8. 2026Drifting Objectives for Refining Discrete Diffusion Language Modelscited by 0
  9. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  10. 2026Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flowcited by 0
  11. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reportedcited by 0
  12. 2026Less Uniform Discrete Diffusion is More Powerful and Scalablecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models
  2. 2026Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
  3. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
  4. 2024Informed Correctors for Discrete Diffusion Models
  5. 2026Optimizers for Diffusion Models: A Controlled Benchmark
  6. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
  7. 2025Diffusion Language Models are Super Data Learners
  8. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps