# Universal Deep Neural Network Compression

arXiv: https://arxiv.org/abs/1802.02271 · HTML: https://mlsanity.com/paper/1802.02271/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2018.

## Built on by (most cited first, 19 of 19)

- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [DKM: Differentiable K-Means Clustering Layer for Neural Network Compression](https://mlsanity.com/paper/2108.12659/) (2021)
- [An Information-Theoretic Justification for Model Pruning](https://mlsanity.com/paper/2102.08329/) (2021)
- [Neural Network Compression for Noisy Storage Devices](https://mlsanity.com/paper/2102.07725/) (2021)
- [Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations](https://mlsanity.com/paper/2110.08271/) (2021)
- [Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training](https://mlsanity.com/paper/2303.13635/) (2023)
- [From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learning](https://mlsanity.com/paper/2501.19067/) (2025)
- [LL-VQ-VAE: Learnable Lattice Vector-Quantization For Efficient Representations](https://mlsanity.com/paper/2310.09382/) (2023)
- [Divide-and-Conquer the NAS puzzle in Resource Constrained Federated Learning Systems](https://mlsanity.com/paper/2305.07135/) (2023)
- [Croesus: Multi-Stage Processing and Transactions for Video-Analytics in Edge-Cloud Systems](https://mlsanity.com/paper/2201.00063/) (2021)
- [Supervised Robustness-preserving Data-free Neural Network Pruning](https://mlsanity.com/paper/2204.00783/) (2022)
- [Coded Deep Learning: Framework and Algorithm](https://mlsanity.com/paper/2501.09849/) (2025)
- [Compact representations of convolutional neural networks via weight pruning and quantization](https://mlsanity.com/paper/2108.12704/) (2021)
- [Deep learning model compression using network sensitivity and gradients](https://mlsanity.com/paper/2210.05111/) (2022)
- [Resource-Limited Automated Ki67 Index Estimation in Breast Cancer](https://mlsanity.com/paper/2401.00014/) (2023)
- [A Novel Structure-Agnostic Multi-Objective Approach for Weight-Sharing Compression in Deep Neural Networks](https://mlsanity.com/paper/2501.03095/) (2025)
- [Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding](https://mlsanity.com/paper/2505.18758/) (2025)
- [Motion-Compensated Weight Compression](https://mlsanity.com/paper/2605.24754/) (2026)
- [When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits](https://mlsanity.com/paper/2610.16020/) (2026)

## Abstract (arXiv)

In this paper, we investigate lossy compression of deep neural networks (DNNs) by weight quantization and lossless source coding for memory-efficient deployment. Whereas the previous work addressed non-universal scalar quantization and entropy coding of DNN weights, we for the first time introduce universal DNN compression by universal vector quantization and universal source coding. In particular, we examine universal randomized lattice quantization of DNNs, which randomizes DNN weights by uniform random dithering before lattice quantization and can perform near-optimally on any source without relying on knowledge of its probability distribution. Moreover, we present a method of fine-tuning vector quantized DNNs to recover the performance loss after quantization. Our experimental results show that the proposed universal DNN compression scheme compresses the 32-layer ResNet (trained on CIFAR-10) and the AlexNet (trained on ImageNet) with compression ratios of 47.1 and 42.5, respectively.