mlsanityData to 14 Oct 2026

Less Uniform Discrete Diffusion is More Powerful and Scalable

arXiv 2609.35817 ↗First posted 21 Sep 2026cs.CL cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform training objective and condition-target confusion during sampling. To address these, we propose Less Uniform Diffusion (LUDI), a novel UDLM framework. Specifically, we (i) introduce a less uniform loss that directs each reverse transition toward the clean token, and (ii) equip the model with per-token time embeddings that supply token-level corruption hints, enabling confidence-based few-step sampling. Experiments across scales show that LUDI yields cleaner supervision and improves few-step generation. We further continue-train a 7B autoregressive model into LUDI-7B, resulting in a UDLM capable of complex reasoning. It achieves a 3-token-per-step speedup over AR decoding and competitive performance compared with masked diffusion baselines, revealing that the full potential of UDLMs for complex generation remains to be unlocked.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 37 of its 47 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

37 of 47 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Stabilizing Reinforcement Learning for Diffusion Language Models6 Mar
  2. 2026Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes4 Mar
  3. 2026The Diffusion Duality, Chapter II: Psi-Samplers24 Feb
  4. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising18 Feb
  5. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models12 Feb
  6. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference28 Dec
  7. 2025Scaling Behavior of Discrete Diffusion Language Models11 Dec
  8. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  9. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs7 Dec
  10. 2025Simple Denoising Diffusion Language Models27 Oct
  11. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Model29 Sep
  12. 2025Dream 7B: Diffusion Large Language Models21 Aug
6 more
  1. 2025The Diffusion Duality12 Jun
  2. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  3. 2024Scaling up Masked Diffusion Models on Text24 Oct
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  6. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  12. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
7 more
  1. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  2. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  3. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  5. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  6. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  7. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Beyond Masked Diffusion Language Models
  2. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
  3. 2026Sumi: Open Uniform Diffusion Language Model from Scratch
  4. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models
  5. 2026Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
  6. 2026ALoDLM: Adaptively Looped Diffusion Language Models
  7. 2026FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version
  8. 2025Encoder-Decoder Diffusion Language Models for Efficient Training and Inference