mlsanityData to 7 Oct 2026

Unlocking the Potential of Diffusion Language Models through Template Infilling

arXiv 2510.13870 ↗First posted 13 Oct 2025cs.CL cs.AIcited by 2

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) have emerged as a promising alternative to Autoregressive Language Models, yet their inference strategies remain limited to prefix-based prompting inherited from the autoregressive paradigm. In this paper, we propose Template Infilling (TI), a tailored conditioning methodology for DLMs. Unlike conventional prefix prompting, TI flexibly aligns structural anchors across the entire target response space, establishing a global blueprint before filling in the masked segments. We demonstrate the effectiveness of our approach on diverse benchmarks, including mathematical reasoning, code generation, and trip planning, achieving consistent improvements of 9.40% over the baseline. Furthermore, we observe that TI provides additional advantages in multi-token generation settings, enabling effective speedup while maintaining generation quality and robustness. By enforcing these global constraints, TI ultimately facilitates System-2 reasoning, empowering the model to deliberate within a structurally defined solution space.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 25 of its 37 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

25 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models18 Aug
  2. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding22 May
  3. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model19 Feb
  4. 2025Theoretical Benefit and Limitation of Diffusion Language Model13 Feb

Other references · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  10. 2023LIMA: Less Is More for Alignmentcited by 1,080
  11. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  12. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
9 more
  1. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  2. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  3. 2021GRAND: Graph Neural Diffusioncited by 258
  4. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  5. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 168
  6. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  7. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 125
  8. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  9. 2025Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient dLLM Inferencecited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026When Confidence Misleads: Suffix Anchoring and Anchor-Proximity Confidence Modulation for Diffusion Language Modelscited by 0
  2. 2026In-Place Instruction Following in Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Diffusion Language Models
  2. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas
  3. 2026Diffusion Language Models: An Experimental Analysis
  4. 2025On Powerful Ways to Generate: Autoregression, Diffusion, and Beyond
  5. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  6. 2026DIP: Dynamic In-Context Planner For Diffusion Language Models
  7. 2026Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning
  8. 2026Introspective Diffusion Language Models