mlsanityData to 14 Oct 2026

Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs

arXiv 2508.14896 ↗First posted 20 Aug 2025cs.CL cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

Recent advances in diffusion large language models (dLLMs) have introduced a promising alternative to autoregressive (AR) LLMs for natural language generation tasks, leveraging full attention and denoising-based decoding strategies. However, the deployment of these models on edge devices remains challenging due to their massive parameter scale and high resource demands. While post-training quantization (PTQ) has emerged as a widely adopted technique for compressing AR LLMs, its applicability to dLLMs remains largely unexplored. In this work, we present the first systematic study on quantizing diffusion-based language models. We begin by identifying the presence of activation outliers, characterized by abnormally large activation values that dominate the dynamic range. These outliers pose a key challenge to low-bit quantization, as they make it difficult to preserve precision for the majority of values. More importantly, we implement state-of-the-art PTQ methods and conduct a comprehensive evaluation across multiple task types and model variants. Our analysis is structured along four key dimensions: bit-width, quantization method, task category, and model type. Through this multi-perspective evaluation, we offer practical insights into the quantization behavior of dLLMs under different configurations. We hope our findings provide a foundation for future research in efficient dLLM deployment. Our code is publicly available at https://github.com/FelixMessi/QDLM.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 89 of its 119 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

89 of 119 references.

References · 89

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2025Qwen3 Technical Reportcited by 8,820
  9. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  10. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
38 more
  1. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  12. 2025Large Language Diffusion Modelscited by 912
  13. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  14. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  15. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  16. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  17. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  18. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  19. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  20. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  21. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  22. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  23. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  24. 2024Massive Activations in Large Language Modelscited by 294
  25. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 285
  26. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  27. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  28. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  29. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  30. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  31. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  32. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 196
  33. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  34. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  35. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  36. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  37. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  38. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 19.

23 papers.

Sort:

By area:

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 37
  2. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 30
  3. 2026Flow caching for autoregressive video generationcited by 25
  4. 2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Modelscited by 12
  5. 2025Hierarchical LoRA MoE for Efficient CTR Model Scalingcited by 10
  6. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  7. 2026Prune as You Generate: Online Rollout Pruning for Faster and Better RLVRcited by 10
  8. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inferencecited by 9
  9. 2026DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantizationcited by 8
  10. 2026Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsificationcited by 5
  11. 2026Subspace Alignment for Vision-Language Model Test-time Adaptationcited by 5
  12. 2026Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncationcited by 5
11 more
  1. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 4
  2. 2026NPU Design for Diffusion Language Model Inferencecited by 3
  3. 2026Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Modelscited by 3
  4. 2025AutoPR: Let's Automate Your Academic Promotion!cited by 2
  5. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Cachingcited by 2
  6. 2026Layer Collapse in Diffusion Language Modelscited by 2
  7. 2026IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generationcited by 2
  8. 2026Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressedcited by 2
  9. 2026FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Modelscited by 1
  10. 2026What does FFN compression change downstream? Same-state causal restoration in diffusion language modelscited by 0
  11. 2026STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
  2. 2026On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
  3. 2026SoloQ: Calibration-Free Quantization for Diffusion Language Models
  4. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  5. 2024Post Training Quantization of Large Language Models with Microscaling Formats
  6. 2026Uncertainty Quantification for Large Language Diffusion Models
  7. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
  8. 2023Efficient Quantization Strategies for Latent Diffusion Models