# Fixed-point optimization of deep neural networks with adaptive step size retraining

arXiv: https://arxiv.org/abs/1702.08171 · HTML: https://mlsanity.com/paper/1702.08171/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2017.

## Built on by (most cited first, 6 of 6)

- [Sparse PointPillars: Maintaining and Exploiting Input Sparsity to Improve Runtime on Embedded Systems](https://mlsanity.com/paper/2106.06882/) (2021)
- [Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms](https://mlsanity.com/paper/2102.02147/) (2021)
- [Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators](https://mlsanity.com/paper/2104.08323/) (2021)
- [Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment](https://mlsanity.com/paper/2202.05048/) (2022)
- [QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applications](https://mlsanity.com/paper/2501.07161/) (2025)
- [USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks](https://mlsanity.com/paper/2412.13724/) (2024)

## Abstract (arXiv)

Fixed-point optimization of deep neural networks plays an important role in hardware based design and low-power implementations. Many deep neural networks show fairly good performance even with 2- or 3-bit precision when quantized weights are fine-tuned by retraining. We propose an improved fixedpoint optimization algorithm that estimates the quantization step size dynamically during the retraining. In addition, a gradual quantization scheme is also tested, which sequentially applies fixed-point optimizations from high- to low-precision. The experiments are conducted for feed-forward deep neural networks (FFDNNs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs).