mlsanityData to 7 Oct 2026

Generalized Interpolating Discrete Diffusion

arXiv 2503.04482 ↗First posted 6 Mar 2025cs.CL cs.AI cs.LGcited by 68

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated tokens. This has prompted exploration of alternative approaches such as discrete diffusion. However, masked diffusion, which has emerged as a popular choice due to its simplicity and effectiveness, reintroduces this inability to revise words. To overcome this, we generalize masked diffusion, deriving a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes. Leveraging a novel diffusion ELBO, we achieve compute-matched state-of-the-art performance in diffusion language modeling. Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality and unlocking the ability for the model to correct its own mistakes, an area where autoregressive models notoriously have struggled. Code: https://github.com/dvruette/gidd/

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 33 of its 52 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

33 of 52 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  2. 2024Scaling up Masked Diffusion Models on Text24 Oct
  3. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  4. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  5. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  7. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  9. 2023Likelihood-Based Diffusion Language Models30 May

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  12. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
12 more
  1. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  4. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  5. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  6. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  7. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  8. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  9. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  10. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
  11. 2023Why Do We Need Weight Decay in Modern Deep Learning?cited by 70
  12. 2019Non-Monotonic Sequential Text Generationcited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 25; 2026: 43.

68 papers.

Sort:

By area:

  1. 2025A Survey on Diffusion Language Modelscited by 75
  2. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  3. 2025Fine-Tuning Masked Diffusion for Provable Self-Correctioncited by 39
  4. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  5. 2025Scaling Behavior of Discrete Diffusion Language Modelscited by 32
  6. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasonercited by 28
  7. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modelingcited by 24
  8. 2025Anchored Diffusion Language Modelcited by 17
  9. 2025Soft-Masked Diffusion Language Modelscited by 15
  10. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 14
  11. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 12
  12. 2026The Diffusion Duality, Chapter II: Psi-Samplerscited by 12
38 more
  1. 2026Continuous Latent Diffusion Language Modelcited by 12
  2. 2025Corrective Diffusion Language Modelscited by 11
  3. 2026Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulationcited by 11
  4. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  5. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
  6. 2025Next Semantic Scale Prediction via Hierarchical Diffusion Language Modelscited by 7
  7. 2025Partition Generative Modeling: Masked Modeling Without Maskscited by 6
  8. 2025Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wallcited by 6
  9. 2026The Design Space of Tri-Modal Masked Diffusion Modelscited by 6
  10. 2025JointDiff: Bridging Continuous and Discrete in Multi-Agent Trajectory Generationcited by 5
  11. 2025Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Modelscited by 5
  12. 2025Guided Star-Shaped Masked Diffusioncited by 5
  13. 2025Masked Diffusion for Generative Recommendationcited by 5
  14. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 5
  15. 2025Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignmentcited by 3
  16. 2025DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generationcited by 3
  17. 2025Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoningcited by 3
  18. 2025Masked Diffusion Models are Secretly Learned-Order Autoregressive Modelscited by 3
  19. 2026ILRR: Inference-Time Steering Method for Masked Diffusion Language Modelscited by 3
  20. 2026Balancing Understanding and Generation in Discrete Diffusion Modelscited by 3
  21. 2026Generalized Discrete Diffusion from Snapshotscited by 3
  22. 2026Neural Continuous-Time Markov Chain: Discrete Diffusion via Decoupled Jump Timing and Directioncited by 3
  23. 2026Provably adaptive sampling with uniform and remasking discrete diffusion modelscited by 3
  24. 2025RIV: Recursive Introspection Mask Diffusion Vision Language Modelcited by 2
  25. 2026Training-Free Self-Correction for Multimodal Masked Diffusion Modelscited by 2
  26. 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillationcited by 2
  27. 2026Towards A Generative Protein Evolution Machine with DPLM-Evocited by 2
  28. 2026Sumi: Open Uniform Diffusion Language Model from Scratchcited by 2
  29. 2026Simplex Relaxation for Discrete Diffusioncited by 2
  30. 2025Deterministic Discrete Denoisingcited by 1
  31. 2025Foundations of Diffusion Models in General State Spaces: A Self-Contained Introductioncited by 1
  32. 2026Generalized Discrete Diffusion with Self-Correctioncited by 1
  33. 2026Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoningcited by 1
  34. 2026Self-Aware Markov Models for Discrete Reasoningcited by 1
  35. 2026Gumbel Distillation for Parallel Text Generationcited by 1
  36. 2026Infinite Mask Diffusion for Few-Step Distillationcited by 1
  37. 2026Re-evaluating Confidence Remasking in Masked Diffusion Language Modelscited by 1
  38. 2026Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processescited by 1

50 most cited of 68 All 68 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Beyond Masked Diffusion Language Models
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
  3. 2024Promises, Outlooks and Challenges of Diffusion Language Modeling
  4. 2026Latent Shadows: The Gaussian-Discrete Duality in Masked Diffusion
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling
  6. 2026Optimizers for Diffusion Models: A Controlled Benchmark
  7. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  8. 2025Optimal Inference Schedules for Masked Diffusion Models