Data to 7 Oct 2026

Learned Step Size Quantization

arXiv 1902.08153 ↗First posted 21 Feb 2019cs.LG stat.MLcited by 604

Not on the map. Why?

Cited by 38 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves the highest accuracy to date on the ImageNet dataset when using models, from a variety of architectures, with weights and activations quantized to 2-, 3- or 4-bits of precision, and that can train 3-bit models that reach full precision baseline accuracy. Our approach builds upon existing methods for learning weights in quantized networks by improving how the quantizer itself is configured. Specifically, we introduce a novel means to estimate and scale the task loss gradient at each weight and activation layer's quantizer step size, such that it can be learned in conjunction with other network parameters. This approach works using different levels of precision as needed for a given system and requires only a simple modification of existing training code.

Key facts. Cited by 604 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 54; 2022: 86; 2023: 109; 2024: 118; 2025: 143; 2026: 94.

604 papers.

Sort:

By area:

2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
2021A White Paper on Neural Network Quantizationcited by 453
2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
2022Post-training Quantization on Diffusion Modelscited by 262
2021I-BERT: Integer-only BERT Quantizationcited by 250

38 more

2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 154
2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
2021Learned Token Pruning for Transformerscited by 138
2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 97
2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformerscited by 91
2022FP8 Quantization: The Power of the Exponentcited by 87
2021ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spottingcited by 84
2021Network Quantization with Element-wise Gradient Scalingcited by 82
2022COIN++: Neural Compression Across Modalitiescited by 81
2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Modelscited by 77
2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 76
2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Modelscited by 67
2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
2023Training Transformers with 4-bit Integerscited by 58
2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
2021Implicit Neural Video Compressioncited by 52

50 most cited of 604 All 604 (JSON)