mlsanityData to 14 Oct 2026

Guided Star-Shaped Masked Diffusion

arXiv 2510.08369 ↗First posted 9 Oct 2025cs.LGcited by 5

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

The performance of pre-trained masked diffusion models is often constrained by their sampling procedure, which makes decisions irreversible and struggles in low-step generation regimes. We introduce a novel sampling algorithm that works with pre-trained models and, after a lightweight fine-tuning of a single layer, significantly improves sample quality and efficiency. Our method reformulates the generation process using a star-shaped paradigm, which inherently allows for error correction. To make this process effective, we augment it with a learnable re-masking scheduler that intelligently identifies and revises likely errors. This approach yields a substantial quality boost, particularly when using a small number of sampling steps. We extensively ablate key components of our approach and show its usability in different scenarios. In comprehensive experiments on text, and code generation, our sampling algorithm outperforms or matches existing methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 56 of its 61 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

56 of 61 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  2. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  5. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling26 Jun
  6. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  7. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  8. 2025Generalized Interpolating Discrete Diffusion6 Mar
  9. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  12. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
8 more
  1. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  2. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  4. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  5. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings29 Feb
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  7. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb
  8. 2022Latent Diffusion for Language Generation19 Dec

Other references · 36

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
24 more
  1. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  2. 2015Teaching Machines to Read and Comprehendcited by 1,049
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  4. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  5. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  6. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  8. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  9. 2024Discrete Flow Matchingcited by 317
  10. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  11. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  12. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  13. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  14. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  15. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  16. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 155
  17. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  18. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  19. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  20. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  21. 2025Edit Flows: Flow Matching with Edit Operationscited by 55
  22. 2023Diffusion Models With Learned Adaptive Noisecited by 51
  23. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 30
  24. 2023Star-Shaped Denoising Diffusion Probabilistic Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Modelscited by 13
  2. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Spacecited by 11
  3. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 3
  4. 2026Progressive Refinement Regulation for Accelerating Diffusion Language Model Decodingcited by 0
  5. 2026Adaptive Order Policies for Masked Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Sampling for Masked Diffusion Models via Information Gain
  2. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking
  3. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models
  4. 2025Parallel Sampling from Masked Diffusion Models via Conditional Independence Testing
  5. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  6. 2025Self-Speculative Masked Diffusions
  7. 2025Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
  8. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generator