# VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantization

arXiv: https://arxiv.org/abs/2005.08501 · HTML: https://mlsanity.com/paper/2005.08501/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 15 of 15)

- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization](https://mlsanity.com/paper/2312.15322/) (2023)
- [Optimizing the Deployment of Tiny Transformers on Low-Power MCUs](https://mlsanity.com/paper/2404.02945/) (2024)
- [HiKonv: High Throughput Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2112.13972/) (2021)
- [Elastic Significant Bit Quantization and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/2109.03513/) (2021)
- [3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration](https://mlsanity.com/paper/2105.06250/) (2021)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applications](https://mlsanity.com/paper/2501.07161/) (2025)
- [Asyn2F: An Asynchronous Federated Learning Framework with Bidirectional Model Aggregation](https://mlsanity.com/paper/2403.01417/) (2024)
- [Minimize Quantization Output Error with Bias Compensation](https://mlsanity.com/paper/2404.01892/) (2024)
- [Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing](https://mlsanity.com/paper/2507.05829/) (2025)
- [Compilation and Optimizations for Efficient Machine Learning on Embedded Systems](https://mlsanity.com/paper/2206.03326/) (2022)
- [HiKonv: Maximizing the Throughput of Quantized Convolution With Novel Bit-wise Management and Computation](https://mlsanity.com/paper/2208.00763/) (2022)
- [AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks](https://mlsanity.com/paper/2304.03782/) (2023)
- [NASH: Neural Architecture Search for Hardware-Optimized Machine Learning Models](https://mlsanity.com/paper/2403.01845/) (2024)

## Abstract (arXiv)

Quantization has been proven to be an effective method for reducing the computing and/or storage cost of DNNs. However, the trade-off between the quantization bitwidth and final accuracy is complex and non-convex, which makes it difficult to be optimized directly. Minimizing direct quantization loss (DQL) of the coefficient data is an effective local optimization method, but previous works often neglect the accurate control of the DQL, resulting in a higher loss of the final DNN model accuracy. In this paper, we propose a novel metric called Vector Loss. Based on this new metric, we develop a new quantization solution called VecQ, which can guarantee minimal direct quantization loss and better model accuracy. In addition, in order to speed up the proposed quantization process during model training, we accelerate the quantization process with a parameterized probability estimation method and template-based derivation calculation. We evaluate our proposed algorithm on MNIST, CIFAR, ImageNet, IMDB movie review and THUCNews text data sets with numerical DNN models. The results demonstrate that our proposed quantization solution is more accurate and effective than the state-of-the-art approaches yet with more flexible bitwidth support. Moreover, the evaluation of our quantized models on Saliency Object Detection (SOD) tasks maintains comparable feature extraction quality with up to 16× weight size reduction.