mlsanityData to 7 Oct 2026

ISQuant: apply squant to the real deployment

arXiv 2407.11037 ↗First posted 5 Jul 2024cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The model quantization technique of deep neural networks has garnered significant attention and has proven to be highly useful in compressing model size, reducing computation costs, and accelerating inference. Many researchers employ fake quantization for analyzing or training the quantization process. However, fake quantization is not the final form for deployment, and there exists a gap between the academic setting and real-world deployment. Additionally, the inclusion of additional computation with scale and zero-point makes deployment a challenging task. In this study, we first analyze why the combination of quantization and dequantization is used to train the model and draw the conclusion that fake quantization research is reasonable due to the disappearance of weight gradients and the ability to approximate between fake and real quantization. Secondly, we propose ISQuant as a solution for deploying 8-bit models. ISQuant is fast and easy to use for most 8-bit models, requiring fewer parameters and less computation. ISQuant also inherits the advantages of SQuant, such as not requiring training data and being very fast at the first level of quantization. Finally We conduct some experiments and found the results is acceptable.our code is available at https://github.com/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 34 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

30 of 34 references.

References · 30

  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  6. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  7. 2019Learned Step Size Quantizationcited by 604
  8. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  9. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  10. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  12. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
18 more
  1. 2021Post-Training Quantization for Vision Transformercited by 274
  2. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  3. 2016Ternary Weight Networkscited by 195
  4. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  5. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  6. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  7. 2020APQ: Joint Search for Network Architecture, Pruning and Quantization Policycited by 98
  8. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 89
  9. 2018Hessian-based Analysis of Large Batch Training and Robustness to Adversariescited by 87
  10. 2018SqueezeNext: Hardware-Aware Neural Network Designcited by 61
  11. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  12. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 51
  13. 2019Any-Precision Deep Neural Networkscited by 39
  14. 2018Post-training 4-bit quantization of convolution networks for rapid-deploymentcited by 38
  15. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  16. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  17. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  18. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26

Built on by

Not cited yet.