mlsanityData to 7 Oct 2026

Hyperspherical Quantization: Toward Smaller and More Accurate Models

arXiv 2212.12653 ↗First posted 24 Dec 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Model quantization enables the deployment of deep neural networks under resource-constrained devices. Vector quantization aims at reducing the model size by indexing model weights with full-precision embeddings, i.e., codewords, while the index needs to be restored to 32-bit during computation. Binary and other low-precision quantization methods can reduce the model size up to 32×, however, at the cost of a considerable accuracy drop. In this paper, we propose an efficient framework for ternary quantization to produce smaller and more accurate compressed models. By integrating hyperspherical learning, pruning and reinitialization, our proposed Hyperspherical Quantization (HQ) method reduces the cosine distance between the full-precision and ternary weights, thus reducing the bias of the straight-through gradient estimator during ternary quantization. Compared with existing work at similar compression levels (~30×, ~40×), our method significantly improves the test accuracy and reduces the model size.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 65 of its 84 references that are in the map. First posted on arXiv 24 Dec 2022.

Builds on

65 of 84 references.

References · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017Mask R-CNNcited by 7,259
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
38 more
  1. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  4. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  5. 2019Learned Step Size Quantizationcited by 604
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2018Rethinking the Value of Network Pruningcited by 595
  8. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  10. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  11. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  12. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  13. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  14. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  15. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  16. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  17. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  18. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  19. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  20. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  21. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  22. 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 212
  23. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  24. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  25. 2016Trained Ternary Quantizationcited by 203
  26. 2016Ternary Weight Networkscited by 195
  27. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 189
  28. 2018Hyperspherical Variational Auto-Encoderscited by 188
  29. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  30. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  31. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  32. 2018Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Searchcited by 122
  33. 2016Hierarchical Multiscale Recurrent Neural Networkscited by 102
  34. 2019And the Bit Goes Down: Revisiting the Quantization of Neural Networkscited by 78
  35. 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 60
  36. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  37. 2017Deep Hyperspherical Learningcited by 55
  38. 2013Estimating or Propagating Gradients Through Stochastic Neuronscited by 54

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 3.

4 papers.

  1. 2023Ternary Quantization: A Surveycited by 2
  2. 2025A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systemscited by 2
  3. 2025SSVQ: Unleashing the Potential of Vector Quantization with Sign-Splittingcited by 0
  4. 2025Extreme Model Compression with Structured Sparsity at Low Precisioncited by 0