mlsanityData to 14 Oct 2026

Compressing Deep Convolutional Networks using Vector Quantization

arXiv 1412.6115 ↗First posted 18 Dec 2014cs.CV cs.LG cs.NEcited by 195

Not on the map. Why?

Abstract (arXiv)

Deep convolutional neural networks (CNN) has become the most promising method for object recognition, repeatedly demonstrating record breaking results for image classification and object detection in recent years. However, a very deep CNN generally involves many layers with millions of parameters, making the storage of the network model to be extremely large. This prohibits the usage of deep CNNs on resource limited hardware, especially cell phones or other embedded devices. In this paper, we tackle this model storage issue by investigating information theoretical vector quantization methods for compressing the parameters of CNNs. In particular, we have found in terms of compressing the most storage demanding dense connected layers, vector quantization methods have a clear gain over existing matrix factorization methods. Simply applying k-means clustering to the weights or conducting product quantization can lead to a very good balance between model size and recognition accuracy. For the 1000-category classification task in the ImageNet challenge, we are able to achieve 16-24 times compression of the network with only 1% loss of classification accuracy using the state-of-the-art CNN.

Key facts. Cited by 195 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Dec 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 54; 2022: 44; 2023: 40; 2024: 29; 2025: 19; 2026: 9.

195 papers.

Sort:

By area:

  1. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  3. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  5. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  6. 2023EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingScited by 196
  7. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
  8. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 175
  9. 2023CompGS: Smaller and Faster Gaussian Splatting with Vector Quantizationcited by 130
  10. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 129
  11. 2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 89
  12. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
38 more
  1. 2022Compressing Volumetric Radiance Fields to 1 MBcited by 68
  2. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  3. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 64
  4. 2021hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devicescited by 62
  5. 2023MixFormerV2: Efficient Fully Transformer Trackingcited by 57
  6. 2021A Survey on Green Deep Learningcited by 52
  7. 2024Tiny Machine Learning: Progress and Futurescited by 40
  8. 2024A Comprehensive Study on Quantization Techniques for Large Language Modelscited by 36
  9. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  10. 2023Visual Tuningcited by 34
  11. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  12. 2025EEG Foundation Models: A Critical Review of Current Progress and Future Directionscited by 32
  13. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  14. 2021Kronecker Decomposition for GPT Compressioncited by 28
  15. 2022A Survey on Federated Recommendation Systemscited by 25
  16. 2022Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networkscited by 24
  17. 2022Data-Efficient Structured Pruning via Submodular Optimizationcited by 23
  18. 2025LVPruning: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Modelscited by 23
  19. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
  20. 2021ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximationscited by 21
  21. 2021KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillationcited by 20
  22. 2023PowerQuant: Automorphism Search for Non-Uniform Quantizationcited by 20
  23. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 20
  24. 2021Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomescited by 19
  25. 2022AdapLeR: Speeding up Inference by Adaptive Length Reductioncited by 19
  26. 2021Stealthy Backdoors as Compression Artifactscited by 18
  27. 2021XtremeDistilTransformers: Task Transfer for Task-agnostic Distillationcited by 18
  28. 2022Self-Knowledge Distillation via Dropoutcited by 18
  29. 2023Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AIcited by 17
  30. 2022SaiT: Sparse Vision Transformers through Adaptive Token Pruningcited by 15
  31. 2021Data-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesiscited by 14
  32. 2021Elbert: Fast Albert with Confidence-Window Based Early Exitcited by 14
  33. 2021Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsitycited by 14
  34. 2022HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaversecited by 14
  35. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  36. 2024Optimal Quantization for Matrix Multiplicationcited by 14
  37. 2022Learning Compressed Embeddings for On-Device Inferencecited by 13
  38. 2022Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruningcited by 13

50 most cited of 195 All 195 (JSON)