mlsanityData to 7 Oct 2026

Softmax Bias Correction for Quantized Generative Models

arXiv 2309.01729 ↗First posted 4 Sep 2023cs.LG cs.AI cs.CVcited by 15

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Post-training quantization (PTQ) is the go-to compression technique for large generative models, such as stable diffusion or large language models. PTQ methods commonly keep the softmax activation in higher precision as it has been shown to be very sensitive to quantization noise. However, this can lead to a significant runtime and power overhead during inference on resource-constraint edge devices. In this work, we investigate the source of the softmax sensitivity to quantization and show that the quantization operation leads to a large bias in the softmax output, causing accuracy degradation. To overcome this issue, we propose an offline bias correction technique that improves the quantizability of softmax without additional compute during deployment, as it can be readily absorbed into the quantization parameters. We demonstrate the effectiveness of our method on stable diffusion v1.5 and 125M-size OPT language model, achieving significant accuracy improvement for 8-bit quantized softmax.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 28 of its 35 references that are in the map. First posted on arXiv 4 Sep 2023.

Builds on

28 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  2. 2023Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations21 Apr
  3. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  4. 2022Post-training Quantization on Diffusion Models28 Nov
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 21

  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  5. 2019Learned Step Size Quantizationcited by 604
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  9. 2021A White Paper on Neural Network Quantizationcited by 453
  10. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  11. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  12. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
9 more
  1. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  2. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  3. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  4. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  5. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  6. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  7. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  8. 2021Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformerscited by 52
  9. 2022Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)cited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 3; 2026: 5.

15 papers.

Sort:

By area:

  1. 2023Clockwork Diffusion: Efficient Generation With Model-Step Distillationcited by 18
  2. 2024Mobile Video Diffusioncited by 17
  3. 2023Efficient Quantization Strategies for Latent Diffusion Modelscited by 16
  4. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  5. 2024RefQSR: Reference-based Quantization for Image Super-Resolution Networkscited by 8
  6. 2025Diffusion Model Quantization: A Reviewcited by 6
  7. 2024Temporal Feature Matters: A Framework for Diffusion Model Quantizationcited by 5
  8. 2024SoftmAP: Software-Hardware Co-design for Integer-Only Softmax on Associative Processorscited by 3
  9. 2026Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Samplingcited by 3
  10. 2025DPQ-HD: Post-Training Compression for Ultra-Low Power Hyperdimensional Computingcited by 2
  11. 2025Text Embedding Knows How to Quantize Text-Guided Diffusion Modelscited by 2
  12. 2026QMC: Efficient SLM Edge Inference via Outlier-Aware Quantization and Emergent Memories Co-Designcited by 1
3 more
  1. 2026A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Modelscited by 1
  2. 2026Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusioncited by 1
  3. 2026Integrated electro-optic attention nonlinearities for transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis
  2. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  3. 2025LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation
  4. 2025Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models
  5. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
  6. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels
  7. 2026SoloQ: Calibration-Free Quantization for Diffusion Language Models
  8. 2026PermuQuant: Lowering Per-Group Quantization Error by Reordering Channels for Diffusion Models