mlsanityData to 7 Oct 2026

Learnable Companding Quantization for Accurate Low-bit Neural Networks

arXiv 2103.07156 ↗First posted 12 Mar 2021cs.CV cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Quantizing deep neural networks is an effective method for reducing memory consumption and improving inference speed, and is thus useful for implementation in resource-constrained devices. However, it is still hard for extremely low-bit models to achieve accuracy comparable with that of full-precision models. To address this issue, we propose learnable companding quantization (LCQ) as a novel non-uniform quantization method for 2-, 3-, and 4-bit models. LCQ jointly optimizes model weights and learnable companding functions that can flexibly and non-uniformly control the quantization levels of weights and activations. We also present a new weight normalization technique that allows more stable training for quantization. Experimental results show that LCQ outperforms conventional state-of-the-art methods and narrows the gap between quantized and full-precision models for image classification and object detection tasks. Notably, the 2-bit ResNet-50 model on ImageNet achieves top-1 accuracy of 75.1% and reduces the gap to 1.7%, allowing LCQ to further exploit the potential of non-uniform quantization.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 12 Mar 2021.

Builds on

22 of 34 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  9. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  10. 2017Focal Loss for Dense Object Detectioncited by 749
  11. 2019Learned Step Size Quantizationcited by 604
  12. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
10 more
  1. 2018Model compression via distillation and quantizationcited by 287
  2. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  3. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  4. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  5. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  6. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  7. 2017Towards Effective Low-bitwidth Convolutional Neural Networkscited by 76
  8. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  9. 2018Deep Neural Network Compression with Single and Multiple Level Quantizationcited by 27
  10. 2019Learning low-precision neural networks without Straight-Through Estimator(STE)cited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 12; 2023: 11; 2024: 6; 2025: 7; 2026: 2.

40 papers.

Sort:

  1. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  2. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 27
  3. 2024Fast Matrix Multiplications for Lookup Table-Quantized LLMscited by 27
  4. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  5. 2022Knowledge Condensation Distillationcited by 20
  6. 2022Wavelet Feature Maps Compression for Image-to-Image CNNscited by 16
  7. 2022CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Spacecited by 13
  8. 2023Quantized Feature Distillation for Network Quantizationcited by 13
  9. 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
  10. 2023DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tablescited by 8
  11. 2023GenQ: Quantization in Low Data Regimes with Generative Synthetic Datacited by 8
  12. 2022SQuAT: Sharpness- and Quantization-Aware Training for BERTcited by 7
28 more
  1. 2023MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantizationcited by 7
  2. 2022Towards Accurate Binary Neural Networks via Modeling Contextual Dependenciescited by 6
  3. 2024Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpartcited by 6
  4. 2025Task Vector Quantization for Memory-Efficient Model Mergingcited by 5
  5. 2022LG-LSQ: Learned Gradient Linear Symmetric Quantizationcited by 2
  6. 2022BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformationcited by 2
  7. 2023Model Compression Methods for YOLOv5: A Reviewcited by 2
  8. 2023Boolean Variation and Boolean Logic BackPropagationcited by 2
  9. 2024HEQuant: Marrying Homomorphic Encryption and Quantization for Communication-Efficient Private Inferencecited by 2
  10. 2024Differentiable Search for Finding Optimal Quantization Strategycited by 2
  11. 2025GDNSQ: Gradual Differentiable Noise Scale Quantization for Low-bit Neural Networkscited by 2
  12. 2025Deep Lookup Networkcited by 2
  13. 2022Combining Gradients and Probabilities for Heterogeneous Approximation of Neural Networkscited by 1
  14. 2022Weight Fixing Networkscited by 1
  15. 2023Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantizationcited by 1
  16. 2024Scheduling Weight Transitions for Quantization-Aware Trainingcited by 1
  17. 2025Coded Deep Learning: Framework and Algorithmcited by 1
  18. 2025Regularization-based Framework for Quantization-, Fault- and Variability-Aware Trainingcited by 1
  19. 2026Boosting Entropy with Bell Box Quantizationcited by 1
  20. 2022Hint-dynamic Knowledge Distillationcited by 0
  21. 2022Error-aware Quantization through Noise Temperingcited by 0
  22. 2023Modular Quantization-Aware Training for 6D Object Pose Estimationcited by 0
  23. 2023Neural Network Compression using Binarization and Few Full-Precision Weightscited by 0
  24. 2023Knowledge Distillation Layer that Lets the Student Decidecited by 0
  25. 2023RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architecturescited by 0
  26. 2025Q2: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantizationcited by 0
  27. 2025Extreme Model Compression with Structured Sparsity at Low Precisioncited by 0
  28. 2026Quantization-Aware Pre-Training with Constrained Empirical Weight Distributioncited by 0