mlsanityData to 13 Oct 2026

Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly

arXiv 2602.00476 ↗First posted 31 Jan 2026cs.LG cs.CLcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models (DLMs) provide a bidirectional generation framework naturally suited for infilling, yet their performance is constrained by the pre-specified infilling length. In this paper, we reveal that DLMs possess an inherent ability to discover the correct infilling length. We identify two key statistical phenomena in the first-step denoising confidence: a local textitOracle Peak that emerges near the ground-truth length and a systematic textitLength Bias that often obscures this signal. By leveraging this signal and calibrating the bias, our training-free method CAL (Calibrated Adaptive Length) enables DLMs to approximate the optimal length through an efficient search before formal decoding. Empirical evaluations demonstrate that CAL improves Pass@1 by up to 47.7% over fixed-length baselines and 40.5% over chat-based adaptive methods in code infilling, while boosting BLEU-2 and ROUGE-L by up to 8.5% and 9.9% in text infilling. These results demonstrate that CAL paves the way for robust DLM infilling without requiring any specialized training. Code is available at https://github.com/NiuHechang/CalibratedAdaptiveLength.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 29 of its 32 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

29 of 32 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
  2. 2025Any-Order Flexible Length Masked Diffusion31 Aug
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  5. 2025Flexible-length Text Infilling for Discrete Diffusion Models16 Jun
  6. 2025Large Language Diffusion Models14 Feb
  7. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  10. 2023A Reparameterized Discrete Diffusion Model for Text Generation11 Feb

Other references · 19

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,884
  4. 2015Character-level Convolutional Networks for Text Classificationcited by 2,319
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,685
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,029
  7. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 570
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 497
  9. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 414
  10. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 275
  11. 2022Efficient Training of Language Models to Fill in the Middlecited by 251
  12. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
7 more
  1. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 123
  2. 2020Enabling Language Models to Fill in the Blankscited by 121
  3. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  4. 2019Text Infillingcited by 32
  5. 2025Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Modelscited by 32
  6. 2025Unveiling the Potential of Diffusion Large Language Model in Controllable Generationcited by 14
  7. 2025Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMscited by 10

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
  2. 2026Length-Adaptive Decoding for Masked Diffusion Machine Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Survival-Guided Length Control for Efficient Diffusion Language Models
  2. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
  3. 2026Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
  4. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
  5. 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models
  6. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  7. 2026Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
  8. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models