mlsanityData to 14 Oct 2026

CDLM: Consistency Diffusion Language Models For Faster Sampling

arXiv 2511.19269 ↗First posted 24 Nov 2025cs.LG cs.CLcited by 21

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) offer a promising parallel generation paradigm but suffer from slow inference due to numerous refinement steps and the inability to use standard KV caching. We introduce CDLM (Consistency Diffusion Language Models), a training-based acceleration method that simultaneously tackles both bottlenecks. CDLM integrates consistency modeling to drastically reduce the number of required sampling steps by enabling multi-token finalization. Furthermore, we enforce a block-wise causal attention mask during fine-tuning, making the model fully compatible with KV caching. Experiments show CDLM achieves 3.6x-14.5x lower latency while maintaining competitive accuracy on math and coding tasks. The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 51 of its 60 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

51 of 60 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  2. 2025dInfer: An Efficient Inference Framework for Diffusion Language Models9 Oct
  3. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  4. 2025Dream 7B: Diffusion Large Language Models21 Aug
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  6. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  7. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  8. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  9. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  10. 2025dKV-Cache: The Cache for Diffusion Language Models21 May
  11. 2025Large Language Diffusion Models14 Feb
  12. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
4 more
  1. 2024Scaling up Masked Diffusion Models on Text24 Oct
  2. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  3. 2022Empowering Diffusion Models on the Embedding Space for Text Generation19 Dec
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  12. 2023Consistency Modelscited by 1,822
23 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  4. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  5. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  7. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  8. 2023Improved Techniques for Training Consistency Modelscited by 439
  9. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 433
  10. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  11. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 251
  12. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 235
  13. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  14. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 139
  15. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124
  16. 2024CLLMs: Consistency Large Language Modelscited by 72
  17. 2025dParallel: Learnable Parallel Decoding for dLLMscited by 65
  18. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusioncited by 57
  19. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 43
  20. 2025QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cachecited by 27
  21. 2025Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Modelscited by 13
  22. 2024Bidirectional Awareness Induction in Autoregressive Seq2Seq Modelscited by 2
  23. 2025UniCMs: A Unified Consistency Model For Efficient Multimodal Generation and Understandingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026DMax: Aggressive Parallel Decoding for dLLMscited by 18
  2. 2026EfficientRollout: System-Aware Self-Speculative Decoding for RL Rolloutscited by 6
  3. 2026Fixed-Point Masked Generative Modelingcited by 5
  4. 2026Swordsman: Entropy-Driven Adaptive Block Partition for Efficient Diffusion Language Modelscited by 3
  5. 2026Few-Step Diffusion Language Models via Trajectory Self-Distillationcited by 3
  6. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Modelscited by 3
  7. 2026MAGE: All-[MASK] Block Already Knows Where to Look in Block Diffusion LLMcited by 2
  8. 2026TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLMcited by 2
  9. 2026VocalNet-MDM: Accelerating Streaming Speech LLM via Self-Distilled Masked Diffusion Modelingcited by 1
  10. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMscited by 1
  11. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  12. 2026CForce: Boosting Parallel Decoding for dLLMs via Consistency Forcingcited by 1
9 more
  1. 2026Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Modelscited by 1
  2. 2026Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Modelscited by 0
  3. 2026HERALD: High-Throughput Block Diffusion LLM Serving via CPU-GPU Cooperative KV Cache Retrievalcited by 0
  4. 2026Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMscited by 0
  5. 2026Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challengescited by 0
  6. 2026OLED-MoE: Accelerating MoE-Based dLLM Inference via Inter-Iteration Locality-Aware Expert Offloadingcited by 0
  7. 2026Alpha Diffusion Language Models: Factorization Alone Is Not the Problemcited by 0
  8. 2026Representation-Space MMD for Diffusion Language Modelscited by 0
  9. 2026Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
  2. 2026CD4LM: Consistency Distillation and aDaptive Decoding for Diffusion Language Models
  3. 2026Consistent Diffusion Language Models
  4. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
  5. 2025Diffusion Language Models are Provably Optimal Parallel Samplers
  6. 2025Sequential Diffusion Language Models
  7. 2026Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models
  8. 2026A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models