# BitQ: Tailoring Block Floating Point Precision for Improved DNN Efficiency on Resource-Constrained Devices

arXiv: https://arxiv.org/abs/2409.17093 · HTML: https://mlsanity.com/paper/2409.17093/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities](https://mlsanity.com/paper/2305.11172/) (2023)
- [Data Movement Is All You Need: A Case Study on Optimizing Transformers](https://mlsanity.com/paper/2007.00072/) (2020)
- [ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation](https://mlsanity.com/paper/2406.18522/) (2024)
- [MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators](https://mlsanity.com/paper/2404.05014/) (2024)
- [PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices](https://mlsanity.com/paper/1909.05073/) (2019)
- [Pruning vs Quantization: Which is Better?](https://mlsanity.com/paper/2307.02973/) (2023)
- [Training DNNs with Hybrid Block Floating Point](https://mlsanity.com/paper/1804.01526/) (2018)
- [FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding](https://mlsanity.com/paper/2110.15456/) (2021)
- [Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization](https://mlsanity.com/paper/2202.00232/) (2022)
- [Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning](https://mlsanity.com/paper/2308.07209/) (2023)
- [Boost Vision Transformer with GPU-Friendly Sparsity and Quantization](https://mlsanity.com/paper/2305.10727/) (2023)
- [FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support](https://mlsanity.com/paper/2203.06673/) (2022)
- [Multiplication-Free Transformer Training via Piecewise Affine Operations](https://mlsanity.com/paper/2305.17190/) (2023)
- [ReFloat: Low-Cost Floating-Point Processing in ReRAM for Accelerating Iterative Linear Solvers](https://mlsanity.com/paper/2011.03190/) (2020)

## Built on by (most cited first, 1 of 1)

- [Low-Precision Streaming PCA](https://mlsanity.com/paper/2510.22440/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) are powerful for cognitive tasks such as image classification, object detection, and scene segmentation. One drawback however is the significant high computational complexity and memory consumption, which makes them unfeasible to run real-time on embedded platforms because of the limited hardware resources. Block floating point (BFP) quantization is one of the representative compression approaches for reducing the memory and computational burden owing to their capability to effectively capture the broad data distribution of DNN models. Unfortunately, prior works on BFP-based quantization empirically choose the block size and the precision that preserve accuracy. In this paper, we develop a BFP-based bitwidth-aware analytical modeling framework (called ``BitQ'') for the best BFP implementation of DNN inference on embedded platforms. We formulate and resolve an optimization problem to identify the optimal BFP block size and bitwidth distribution by the trade-off of both accuracy and performance loss. Experimental results show that compared with an equal bitwidth setting, the BFP DNNs with optimized bitwidth allocation provide efficient computation, preserving accuracy on famous benchmarks. The source code and data are available at https://github.com/Cheliosoops/BitQ.