# Deep Neural Network Compression with Single and Multiple Level Quantization

arXiv: https://arxiv.org/abs/1803.03289 · HTML: https://mlsanity.com/paper/1803.03289/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2018.

## Built on by (most cited first, 27 of 27)

- [SqueezeLLM: Dense-and-Sparse Quantization](https://mlsanity.com/paper/2306.07629/) (2023)
- [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://mlsanity.com/paper/2110.13981/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning](https://mlsanity.com/paper/2110.08394/) (2021)
- [Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Framework](https://mlsanity.com/paper/2107.12422/) (2021)
- [FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding](https://mlsanity.com/paper/2109.03787/) (2021)
- [Learnable Companding Quantization for Accurate Low-bit Neural Networks](https://mlsanity.com/paper/2103.07156/) (2021)
- [LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks](https://mlsanity.com/paper/2111.08988/) (2021)
- [Context-Aware Streaming Perception in Dynamic Environments](https://mlsanity.com/paper/2208.07479/) (2022)
- [Low-rank Tensor Decomposition for Compression of Convolutional Neural Networks Using Funnel Regularization](https://mlsanity.com/paper/2112.03690/) (2021)
- [SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization](https://mlsanity.com/paper/2412.04180/) (2024)
- [HEMP: High-order Entropy Minimization for neural network comPression](https://mlsanity.com/paper/2107.05298/) (2021)
- [Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks](https://mlsanity.com/paper/2409.02134/) (2024)
- [Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size](https://mlsanity.com/paper/2503.04704/) (2025)
- [Model compression as constrained optimization, with application to neural nets. Part V: combining compressions](https://mlsanity.com/paper/2107.04380/) (2021)
- [GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models](https://mlsanity.com/paper/2501.12956/) (2025)
- [One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training](https://mlsanity.com/paper/2401.16760/) (2024)
- [Deep Lookup Network](https://mlsanity.com/paper/2509.13662/) (2025)
- [Croesus: Multi-Stage Processing and Transactions for Video-Analytics in Edge-Cloud Systems](https://mlsanity.com/paper/2201.00063/) (2021)
- [STN: Scalable Tensorizing Networks via Structure-Aware Training and Adaptive Compression](https://mlsanity.com/paper/2205.15198/) (2022)
- [On Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guarantee](https://mlsanity.com/paper/2303.06815/) (2023)
- [Automated Heterogeneous Low-Bit Quantization of Multi-Model Deep Learning Inference Pipeline](https://mlsanity.com/paper/2311.05870/) (2023)
- [Neuronal Group Communication for Efficient Neural representation](https://mlsanity.com/paper/2510.16851/) (2025)
- [SigmaQuant: Hardware-Aware Heterogeneous Quantization Method for Edge DNN Inference](https://mlsanity.com/paper/2602.22136/) (2026)
- [LAB: Learnable Activation Binarizer for Binary Neural Networks](https://mlsanity.com/paper/2210.13858/) (2022)
- [On GNN explanability with activation rules](https://mlsanity.com/paper/2406.11594/) (2024)
- [CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts](https://mlsanity.com/paper/2604.10496/) (2026)

## Abstract (arXiv)

Network quantization is an effective solution to compress deep neural networks for practical usage. Existing network quantization methods cannot sufficiently exploit the depth information to generate low-bit compressed network. In this paper, we propose two novel network quantization approaches, single-level network quantization (SLQ) for high-bit quantization and multi-level network quantization (MLQ) for extremely low-bit quantization (ternary).We are the first to consider the network quantization from both width and depth level. In the width level, parameters are divided into two parts: one for quantization and the other for re-training to eliminate the quantization loss. SLQ leverages the distribution of the parameters to improve the width level. In the depth level, we introduce incremental layer compensation to quantize layers iteratively which decreases the quantization loss in each iteration. The proposed approaches are validated with extensive experiments based on the state-of-the-art neural networks including AlexNet, VGG-16, GoogleNet and ResNet-18. Both SLQ and MLQ achieve impressive results.