mlsanityData to 14 Oct 2026

Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Way

arXiv 2510.24605 ↗First posted 28 Oct 2025cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Diffusion-based large language models (dLLMs) have exhibited substantial potential for parallel text generation, which may enable more efficient generation compared to autoregressive models. However, current dLLMs suffer from fixed generation lengths, which indicates the generation lengths of dLLMs have to be determined before decoding as a hyper-parameter, leading to issues in efficiency and flexibility. To solve these problems, in this work, we propose to train a diffusion LLM with native variable generation lengths, abbreviated as dLLM-Var. Concretely, we aim to train a model to accurately predict the [EOS] token in the generated text, which makes a dLLM be able to natively infer in a block diffusion manner, while still maintaining the ability of global bi-directional (full) attention and high parallelism. Experiments on standard benchmarks demonstrate that our method achieves a 30.1x speedup over traditional dLLM inference paradigms and a 2.4x speedup relative to autoregressive models such as Qwen and Llama. Our method achieves higher accuracy and faster inference, elevating dLLMs beyond mere academic novelty and supporting their practical use in real-world applications. Codes and models have been released.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

18 of 21 references.

References · 18

  1. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  2. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  3. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 360
  4. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scalingcited by 189
  6. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  7. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  8. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  9. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  10. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  11. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 129
  12. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcingcited by 116
6 more
  1. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  2. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  3. 2025Any-Order Flexible Length Masked Diffusioncited by 51
  4. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Modelscited by 37
  5. 2025dInfer: An Efficient Inference Framework for Diffusion Language Modelscited by 37
  6. 2025d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Cachingcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 20.

22 papers.

Sort:

By area:

  1. 2026Improved Large Language Diffusion Modelscited by 14
  2. 2025SDAR-VL: Stable and Efficient Block-wise Diffusion for Vision-Language Understandingcited by 13
  3. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 13
  4. 2026LLaDA-o: An Effective and Length-Adaptive Omni Diffusion Modelcited by 10
  5. 2026ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMscited by 7
  6. 2026DSB: Dynamic Sliding Block Scheduling for Diffusion LLMscited by 7
  7. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  8. 2026The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Checkcited by 6
  9. 2025Orchestrating Dual-Boundaries: An Arithmetic Intensity Inspired Acceleration Framework for Diffusion Language Modelscited by 5
  10. 2026Diffusion Language Models Are Natively Length-Awarecited by 5
  11. 2026Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncationcited by 5
  12. 2026Improving Variable-Length Generation in Diffusion Language Models via Length Regularizationcited by 4
10 more
  1. 2026Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Modelscited by 3
  2. 2026Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Modelscited by 3
  3. 2026Plan, Verify and Fill: A Structured Parallel Decoding Approach for Diffusion Language Modelscited by 2
  4. 2026DOS: Dependency-Oriented Sampler for Masked Diffusion Language Modelscited by 2
  5. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  6. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  7. 2026Diffusion Large Language Models for Visual Speech Recognitioncited by 0
  8. 2026Length-Adaptive Decoding for Masked Diffusion Machine Translationcited by 0
  9. 2026DRRG: A Discrete Diffusion Framework for Radiology Report Generationcited by 0
  10. 2026Routing by Reasoning Need: Trajectory-Aware Decoding Control for Diffusion Vision-Language Modelscited by 0