mlsanityData to 14 Oct 2026

ρ-textttEOS: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs

arXiv 2601.22527 ↗First posted 30 Jan 2026cs.CLcited by 7

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Beyond parallel generation and global context modeling, current masked diffusion large language models (masked dLLMs, i.e., LLaDA) suffer from a fundamental limitation: they require a predefined, fixed generation length, which lacks flexibility and forces an inevitable trade-off between output quality and computational efficiency. To address this, we study the denoising dynamics and find that the implicit density (ρ) of end-of-sequence (textttEOS) tokens serves as a reliable signal of generation sufficiency. In particular, the evolving implicit textttEOS density during denoising reveals whether the current masked space is excessive or insufficient, thereby guiding the adjustment direction for generation length. Building on this insight, we propose textbfρ-textttEOS, a training-free, single-stage strategy that enables bidirectional variable-length generation for masked dLLMs. Unlike prior two-stage approaches--which require separate length adjustment and iterative mask insertion phases while supporting only unidirectional expansion--textbfρ-textttEOS achieves bidirectional length adjustment within a unified denoising process by continuously estimating the implicit textttEOS density: excessively high density triggers textttMASK token contraction, while insufficient density induces expansion. Extensive experiments on mathematics and code benchmarks demonstrate that textbfρ-textttEOS achieves comparable performance while substantially improving inference efficiency and token utilization. Code is available at https://github.com/yjyddq/rho-EOS.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 29 of its 33 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

29 of 33 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference28 Dec
  2. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models10 Oct
  3. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  4. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Model29 Sep
  5. 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step28 Sep
  6. 2025Any-Order Flexible Length Masked Diffusion31 Aug
  7. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  8. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  9. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  10. 2025Large Language Diffusion Models14 Feb
  11. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  12. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
3 more
  1. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  2. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  3. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  11. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  12. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
2 more
  1. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 34
  2. 2025Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Waycited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executorcited by 5
  2. 2026Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Modelscited by 3
  3. 2026VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Terminationcited by 1
  4. 2026Expanding Flow Mapscited by 1
  5. 2026A Continuous-Time Markov Chain Framework for Insertion Language Modelscited by 0
  6. 2026Length-Adaptive Decoding for Masked Diffusion Machine Translationcited by 0
  7. 2026CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  2. 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models
  3. 2026Informed Masking: Structure-Aware Perturbation for Reinforcement Learning in Diffusion Large Language Models
  4. 2026On Trajectory-Aware Training for Masked Diffusion Language Models
  5. 2026Mask Is What DLLM Needs: A Masked Data Training Paradigm for Diffusion LLMs
  6. 2026SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
  7. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMs
  8. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs