mlsanityData to 7 Oct 2026

Loss-aware Weight Quantization of Deep Networks

arXiv 1802.08635 ↗First posted 23 Feb 2018cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

The huge size of deep networks hinders their use in small computing devices. In this paper, we consider compressing the network by weight quantization. We extend a recently proposed loss-aware weight binarization scheme to ternarization, with possibly different scaling parameters for the positive and negative weights, and m-bit (where m > 2) quantization. Experiments on feedforward and recurrent neural networks show that the proposed scheme outperforms state-of-the-art weight quantization algorithms, and is as accurate (or even more accurate) than the full-precision network.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 12; 2023: 3; 2024: 4; 2025: 2; 2026: 1.

31 papers.

Sort:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  3. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  4. 2021Trustworthy AI: A Computational Perspectivecited by 93
  5. 2022Compression of Generative Pre-trained Language Models via Quantizationcited by 79
  6. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
  7. 2022CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolutioncited by 26
  8. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  9. 2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantizationcited by 23
  10. 2022EcoFormer: Energy-Saving Attention with Linear Complexitycited by 21
  11. 2021Distance-aware Quantizationcited by 19
  12. 2022Exploring Extreme Parameter Compression for Pre-trained Language Modelscited by 15
19 more
  1. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
  2. 2022Adaptive Low-Precision Training for Embeddings in Click-Through Rate Predictioncited by 11
  3. 2022QReg: On Regularization Effects of Quantizationcited by 9
  4. 2022Class-based Quantization for Neural Networkscited by 9
  5. 2021Adaptive Precision Training (AdaPT): A dynamic fixed point quantized training approach for DNNscited by 8
  6. 2021Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Acceleratorscited by 6
  7. 2023Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networkscited by 4
  8. 2025Radio: Rate-Distortion Optimization for Large Language Model Compressioncited by 4
  9. 2024The Binary Quantized Neural Network for Dense Prediction via Specially Designed Upsampling and Attentioncited by 3
  10. 2022FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representationcited by 2
  11. 2022AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networkscited by 2
  12. 2023Ternary Quantization: A Surveycited by 2
  13. 2024One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Trainingcited by 2
  14. 2024Foundations of Large Language Model Compression -- Part 1: Weight Quantizationcited by 2
  15. 2021NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantizationcited by 0
  16. 2022A Closer Look at Hardware-Friendly Weight Quantizationcited by 0
  17. 2022Hyperspherical Loss-Aware Ternary Quantizationcited by 0
  18. 2023Learning to Learn with Indispensable Connectionscited by 0
  19. 2026DiBA: Diagonal and Binary Matrix Approximation for Neural Network Weight Compressioncited by 0