# Fixed Point Quantization of Deep Convolutional Networks

arXiv: https://arxiv.org/abs/1511.06393 · HTML: https://mlsanity.com/paper/1511.06393/

## Key facts

- Cited by 143 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2015.

## Built on by (most cited first, 30 of 143)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Knowledge distillation: A good teacher is patient and consistent](https://mlsanity.com/paper/2106.05237/) (2021)
- [Understanding and Overcoming the Challenges of Efficient Transformer Quantization](https://mlsanity.com/paper/2109.12948/) (2021)
- [A Survey on Large Language Model Acceleration based on KV Cache Management](https://mlsanity.com/paper/2412.19442/) (2024)
- [LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models](https://mlsanity.com/paper/2206.09557/) (2022)
- [FP8 Quantization: The Power of the Exponent](https://mlsanity.com/paper/2208.09225/) (2022)
- [Federated Learning Enables Big Data for Rare Cancer Boundary Detection](https://mlsanity.com/paper/2204.10836/) (2022)
- [Pruning vs Quantization: Which is Better?](https://mlsanity.com/paper/2307.02973/) (2023)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization](https://mlsanity.com/paper/2202.05239/) (2022)
- [Efficient Video Compression via Content-Adaptive Super-Resolution](https://mlsanity.com/paper/2104.02322/) (2021)
- [Adaptive Data-Free Quantization](https://mlsanity.com/paper/2303.06869/) (2023)
- [Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme](https://mlsanity.com/paper/2110.14439/) (2021)
- [Hardly Perceptible Trojan Attack against Neural Networks with Bit Flips](https://mlsanity.com/paper/2207.13417/) (2022)
- [It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher](https://mlsanity.com/paper/2203.17008/) (2022)
- [Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Models](https://mlsanity.com/paper/2305.12356/) (2023)
- [DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos](https://mlsanity.com/paper/2203.03996/) (2022)
- [Secure Quantized Training for Deep Learning](https://mlsanity.com/paper/2107.00501/) (2021)
- [Patch-wise Mixed-Precision Quantization of Vision Transformer](https://mlsanity.com/paper/2305.06559/) (2023)
- [Neural Network Quantization for Efficient Inference: A Survey](https://mlsanity.com/paper/2112.06126/) (2021)
- [DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware](https://mlsanity.com/paper/2207.00083/) (2022)
- [One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training](https://mlsanity.com/paper/2308.07934/) (2023)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [Rethinking Data-Free Quantization as a Zero-Sum Game](https://mlsanity.com/paper/2302.09572/) (2023)
- [Dynamic Precision Analog Computing for Neural Networks](https://mlsanity.com/paper/2102.06365/) (2021)
- [Low-Precision Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2206.09909/) (2022)
- [Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices](https://mlsanity.com/paper/2101.04446/) (2021)
- [How RL Agents Behave When Their Actions Are Modified](https://mlsanity.com/paper/2102.07716/) (2021)
- [GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity](https://mlsanity.com/paper/2108.11033/) (2021)
- [Multi-task Learning Approach for Modulation and Wireless Signal Classification for 5G and Beyond: Edge Deployment via Model Compression](https://mlsanity.com/paper/2203.00517/) (2022)

## Abstract (arXiv)

In recent years increasingly complex architectures for deep convolution networks (DCNs) have been proposed to boost the performance on image recognition tasks. However, the gains in performance have come at a cost of substantial increase in computation and model storage resources. Fixed point implementation of DCNs has the potential to alleviate some of these complexities and facilitate potential deployment on embedded hardware. In this paper, we propose a quantizer design for fixed point implementation of DCNs. We formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers. Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer >20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark.