# Performance Guaranteed Network Acceleration via High-Order Residual Quantization

arXiv: https://arxiv.org/abs/1708.08687 · HTML: https://mlsanity.com/paper/1708.08687/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2017.

## Built on by (most cited first, 26 of 26)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Autoregressive Image Generation using Residual Quantization](https://mlsanity.com/paper/2203.01941/) (2022)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [Extreme Compression of Large Language Models via Additive Quantization](https://mlsanity.com/paper/2401.06118/) (2024)
- [BiLLM: Pushing the Limit of Post-Training Quantization for LLMs](https://mlsanity.com/paper/2402.04291/) (2024)
- [A comprehensive review of Binary Neural Network](https://mlsanity.com/paper/2110.06804/) (2021)
- [AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets](https://mlsanity.com/paper/2208.08084/) (2022)
- [Distribution-sensitive Information Retention for Accurate Binary Neural Network](https://mlsanity.com/paper/2109.12338/) (2021)
- [QVGen: Pushing the Limit of Quantized Video Generative Models](https://mlsanity.com/paper/2505.11497/) (2025)
- ["BNN - BN = ?": Training Binary Neural Networks without Batch Normalization](https://mlsanity.com/paper/2104.08215/) (2021)
- [Lipschitz Continuity Retained Binary Neural Network](https://mlsanity.com/paper/2207.06540/) (2022)
- [BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models](https://mlsanity.com/paper/2404.05662/) (2024)
- [Q-LIC: Quantizing Learned Image Compression with Channel Splitting](https://mlsanity.com/paper/2205.14510/) (2022)
- [Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models](https://mlsanity.com/paper/2302.04464/) (2023)
- [Equal Bits: Enforcing Equally Distributed Binary Network Weights](https://mlsanity.com/paper/2112.03406/) (2021)
- [Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks](https://mlsanity.com/paper/2110.09195/) (2021)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Single-path Bit Sharing for Automatic Loss-aware Model Compression](https://mlsanity.com/paper/2101.04935/) (2021)
- [Self-Distribution Binary Neural Networks](https://mlsanity.com/paper/2103.02394/) (2021)
- [End-to-End Learned Image Compression with Quantized Weights and Activations](https://mlsanity.com/paper/2111.09348/) (2021)
- [BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance](https://mlsanity.com/paper/2211.06987/) (2022)
- [HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks](https://mlsanity.com/paper/2205.09839/) (2022)
- [Searching Similarity Measure for Binarized Neural Networks](https://mlsanity.com/paper/2206.03325/) (2022)
- [IR2Net: Information Restriction and Information Recovery for Accurate Binary Neural Networks](https://mlsanity.com/paper/2210.02637/) (2022)
- [AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks](https://mlsanity.com/paper/2304.03782/) (2023)
- [DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs](https://mlsanity.com/paper/2306.15390/) (2023)

## Abstract (arXiv)

Input binarization has shown to be an effective way for network acceleration. However, previous binarization scheme could be regarded as simple pixel-wise thresholding operations (i.e., order-one approximation) and suffers a big accuracy loss. In this paper, we propose a highorder binarization scheme, which achieves more accurate approximation while still possesses the advantage of binary operation. In particular, the proposed scheme recursively performs residual quantization and yields a series of binary input images with decreasing magnitude scales. Accordingly, we propose high-order binary filtering and gradient propagation operations for both forward and backward computations. Theoretical analysis shows approximation error guarantee property of proposed method. Extensive experimental results demonstrate that the proposed scheme yields great recognition accuracy while being accelerated.