mlsanityData to 14 Oct 2026

From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs

arXiv 2512.06776 ↗First posted 7 Dec 2025cs.CL cs.AIcited by 24

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) enable fast generation, yet training large DLMs from scratch is costly. As a practical shortcut, adapting off-the-shelf Auto-Regressive (AR) model weights into a DLM could quickly equip the DLM with strong long-context generation capabilies. Prior "adaptation" attempts either modify logits or randomly grow attention masks to Full-Sequence diffusion, or simply transplant AR weights into a Block-Diffusion recipe, leaving two key questions unaddressed: where is the final destination of adaptation, and how to adapt better? For manifold benefits, we reframe the whole AR-to-DLM adaptation under the Block-Diffusion paradigm, transitioning from block size 1 to the final Block-Diffusion state. Concretely, the principled pathway of adaptation is designed as follows: we keep a context-causal path where causal attention is kept in the prefix, an efficient parallel adaptation procedure where an AR guidance is maintained, and gradual increment of the generation block size for a smoother transition. Built on these components, the adaptation is proved competitive on various models at different scales. With better adaptation, we propose NBDiff-7B that could inherit the long-context modeling and reasoning capabilities, and achieve state-of-the-art performance among the 7B-class DLMs. Codes: https://github.com/YuchuanTian/NBDiff.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 30 references that are in the map. First posted on arXiv 7 Dec 2025.

Builds on

27 of 30 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  2. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation25 Sep
  3. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  4. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  5. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  6. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  7. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  8. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  9. 2025Large Language Diffusion Models14 Feb
  10. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  11. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
2 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  2. 2024Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models12 Feb

Other references · 13

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  11. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  12. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
1 more
  1. 2025Thinking Inside the Mask: In-Place Prompting in Diffusion LLMscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 23.

24 papers.

Sort:

By area:

  1. 2025WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inferencecited by 41
  2. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  3. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 13
  4. 2026Deferred Commitment Decoding for Diffusion Language Modelscited by 12
  5. 2026Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrowcited by 11
  6. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  7. 2026TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Accelerationcited by 8
  8. 2026Introspective Diffusion Language Modelscited by 8
  9. 2026Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decodingcited by 7
  10. 2026When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Modelscited by 7
  11. 2026DLLM Agent: See Farther, Run Fastercited by 6
  12. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 4
12 more
  1. 2026Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusioncited by 4
  2. 2026Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variantscited by 3
  3. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  4. 2026Locally Coherent Parallel Decoding in Diffusion Language Modelscited by 3
  5. 2026BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuningcited by 3
  6. 2026Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafterscited by 3
  7. 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillationcited by 2
  8. 2026Rethinking the Generation Order of Block Diffusion Language Modelscited by 2
  9. 2026d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Modelscited by 0
  10. 2026Less Uniform Discrete Diffusion is More Powerful and Scalablecited by 0
  11. 2026ALoDLM: Adaptively Looped Diffusion Language Modelscited by 0
  12. 2026Towards Unbiased On-Policy Distillation for Block Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  2. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  3. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
  4. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model
  5. 2025CoDA: Coding LM via Diffusion Adaptation
  6. 2026Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
  7. 2026Adaptation to Intrinsic Dependence in Diffusion Language Models
  8. 2026Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models