mlsanityData to 7 Oct 2026

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

arXiv 1807.10029 ↗First posted 26 Jul 2018cs.CV cs.AIcited by 272

Not on the map. Why?

Abstract (arXiv)

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets

Key facts. Cited by 272 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 85; 2022: 70; 2023: 52; 2024: 27; 2025: 33; 2026: 5.

272 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2022Efficiently Scaling Transformer Inferencecited by 529
  4. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  5. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  6. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  7. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  8. 2023EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingScited by 191
  9. 2022FP8 Formats for Deep Learningcited by 187
  10. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  11. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  12. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
38 more
  1. 2021Network Quantization with Element-wise Gradient Scalingcited by 82
  2. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  3. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  4. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  5. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  6. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  7. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 62
  8. 2021ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Trainingcited by 61
  9. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  10. 2023Training Transformers with 4-bit Integerscited by 58
  11. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  12. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  13. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  14. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  15. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
  16. 2021ReCU: Reviving the Dead Weights in Binary Neural Networkscited by 51
  17. 2022BiT: Robustly Binarized Multi-distilled Transformercited by 51
  18. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 44
  19. 2021A comprehensive review of Binary Neural Networkcited by 44
  20. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  21. 2022PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformerscited by 42
  22. 2021Learnable Companding Quantization for Accurate Low-bit Neural Networkscited by 40
  23. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  24. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  25. 2021BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantizationcited by 37
  26. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 37
  27. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  28. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  29. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  30. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 32
  31. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  32. 2021DKM: Differentiable K-Means Clustering Layer for Neural Network Compressioncited by 31
  33. 2021Auto-Split: A General Framework of Collaborative Edge-Cloud AIcited by 29
  34. 2023Efficient Post-training Quantization with FP8 Formatscited by 29
  35. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  36. 2021Distribution Adaptive INT8 Quantization for Training CNNscited by 27
  37. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 27
  38. 2024Fast Matrix Multiplications for Lookup Table-Quantized LLMscited by 27

50 most cited of 272 All 272 (JSON)