mlsanityData to 14 Oct 2026

CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcing

arXiv 2608.13925 ↗First posted 14 Aug 2026cs.LG cs.AI cs.CLcited by 1

On the map: Efficient LLM inference and serving → Diffusion LLM and MoE inference

Abstract (arXiv)

Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass. However, existing dLLMs can suffer from unreliable predictions in early denoising stages under aggressive parallelism strategies, leading to errors that can propagate to later stages. To tackle this issue, we present Consistency Forcing (CForce) for dLLMs, a distillation method to force the mask predictions of early stages to align with those of later stages. CForce trains the model on pre-collected self-rollout trajectories, thereby improving training-inference alignment. We introduce Confidence Adaptive KL Divergence as a distillation objective to conjoin the merits of forward and reverse KL. We further provide a theoretical analysis for the consistency objective to explain why CForce can approximately minimize the prediction error of early stages. Critically, the same formulation applies to both mask-to-token decoding and edit-capable decoding; in the edit-capable case, later token-to-token refinements provide additional supervision for earlier masked-state predictions. Experiments on non-edit and edit-capable LLaDA models show improved speed-quality trade-offs, especially under high-parallelism decoding budgets. Code is available at: https://github.com/inclusionAI/dFactory.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion LLM and MoE inference, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 42 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

21 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026DMax: Aggressive Parallel Decoding for dLLMs9 Apr
  2. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation12 Jan
  3. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding28 May

Other references · 17

  1. 2023Let's Verify Step by Stepcited by 3,789
  2. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  6. 2022On Distillation of Guided Diffusion Modelscited by 725
  7. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 348
  8. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
  9. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  10. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  11. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 166
  12. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
5 more
  1. 2025The Diffusion Dualitycited by 135
  2. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 60
  3. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Modelscited by 47
  4. 2025CDLM: Consistency Diffusion Language Models For Faster Samplingcited by 21
  5. 2024See Further When Clear: Curriculum Consistency Modelcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025dParallel: Learnable Parallel Decoding for dLLMs
  2. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models
  3. 2026Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding
  4. 2026DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
  5. 2026LEAP: Unlocking dLLM Parallelism via Lookahead Early-Convergence Token Detection
  6. 2026Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
  7. 2026Stability-Weighted Decoding for Diffusion Language Models
  8. 2025Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding