mlsanityData to 7 Oct 2026

On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks

arXiv 2604.20079 ↗First posted 22 Apr 2026cs.LG cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Auto-regressive Large Language Models (LLMs) achieve strong performance on coding tasks, but incur high memory and inference costs. Diffusion-based language models (d-LLMs) offer bounded inference cost via iterative denoising, but their behavior under post-training quantization (PTQ) has been sparsely explored. We investigate the application and robustness of PTQ techniques, specifically GPTQ and a modified Hessian-Aware Quantization (HAWQ) algorithm, on a diffusion-based coding LLM (CoDA) and observe that these methods applied to CoDA exhibit greater robustness at low bitwidths compared to Qwen3-1.7B, its auto-regressive counterpart, under a standardized evaluation pipeline. We find that in our setup, CoDA exhibits greater robustness at low bitwidths (2-4 bits), with smaller accuracy degradation across HumanEval and MBPP benchmarks. Additionally, mixed-precision configurations derived from HAWQ provide smooth trade-offs across accuracy, latency, and memory. The results suggest that diffusion LLMs may offer advantages for efficient deployment due to more quantization-resilience.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 38 of its 47 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

38 of 47 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025CoDA: Coding LM via Diffusion Adaptation27 Sep
  2. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  3. 2025Large Language Diffusion Models14 Feb
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2025Qwen3 Technical Reportcited by 8,195
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2016Pointer Sentinel Mixture Modelscited by 2,992
21 more
  1. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  8. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  9. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  10. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  11. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  12. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  13. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  14. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  15. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  16. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  17. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  18. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  19. 2024Scaling Laws for Precisioncited by 92
  20. 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 79
  21. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 48

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SoloQ: Calibration-Free Quantization for Diffusion Language Models
  2. 2026Uncertainty Quantification for Large Language Diffusion Models
  3. 2026Diffusion Language Models: An Experimental Analysis
  4. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models
  5. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis
  6. 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
  7. 2025A Survey on Diffusion Language Models
  8. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion