mlsanityData to 14 Oct 2026

Reducing Storage of Pretrained Neural Networks by Rate-Constrained Quantization and Entropy Coding

arXiv 2505.18758 ↗First posted 24 May 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The ever-growing size of neural networks poses serious challenges on resource-constrained devices, such as embedded sensors. Compression algorithms that reduce their size can mitigate these problems, provided that model performance stays close to the original. We propose a novel post-training compression framework that combines rate-aware quantization with entropy coding by (1) extending the well-known layer-wise loss by a quadratic rate estimation, and (2) providing locally exact solutions to this modified objective following the Optimal Brain Surgeon (OBS) method. Our method allows for very fast decoding and is compatible with arbitrary quantization grids. We verify our results empirically by testing on various computer-vision networks, achieving a 20-40% decrease in bit rate at the same performance as the popular compression algorithm NNCodec. Our code is available at https://github.com/Conzel/cerwu.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 55 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

36 of 55 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2019Searching for MobileNetV3cited by 1,994
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  9. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  10. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  12. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
24 more
  1. 2021A White Paper on Neural Network Quantizationcited by 472
  2. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  3. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  4. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  6. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  7. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  8. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  9. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  10. 2023Memory Efficient Optimizers with 4-bit Statescited by 54
  11. 2024Accurate LoRA-Finetuning Quantization of LLMs via Information Retentioncited by 41
  12. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  13. 2019DeepCABAC: A Universal Compression Algorithm for Deep Neural Networkscited by 26
  14. 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiencescited by 21
  15. 2018Universal Deep Neural Network Compressioncited by 19
  16. 2024NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networkscited by 14
  17. 2022Understanding Entropy Coding With Asymmetric Numeral Systems (ANS): a Statistician's Perspectivecited by 12
  18. 2018Compact and Computationally Efficient Representation of Deep Neural Networkscited by 11
  19. 2019Feature Map Transform Coding for Energy-Efficient CNN Inferencecited by 11
  20. 2025Compression Scaling Laws:Unifying Sparsity and Quantizationcited by 11
  21. 2024Inference Optimization of Foundation Models on AI Acceleratorscited by 9
  22. 2019CAT: Compression-Aware Training for bandwidth reductioncited by 8
  23. 2020PENNI: Pruned Kernel Sharing for Efficient CNN Inferencecited by 6
  24. 2021Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Neural Compression with Inference-time Decoding
  2. 2021HEMP: High-order Entropy Minimization for neural network comPression
  3. 2024Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding
  4. 2023Optimized learned entropy coding parameters for practical neural-based image and video compression
  5. 2025Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization
  6. 2025MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training
  7. 2023Bandwidth-efficient Inference for Neural Image Compression
  8. 2025Exploiting Latent Properties to Optimize Neural Codecs