# Realizing Unaligned Block-wise Pruning for DNN Acceleration on Mobile Devices

arXiv: https://arxiv.org/abs/2407.19644 · HTML: https://mlsanity.com/paper/2407.19644/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://mlsanity.com/paper/1707.06342/) (2017)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Accelerating Sparse Deep Neural Networks](https://mlsanity.com/paper/2104.08378/) (2021)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [Fast Sparse ConvNets](https://mlsanity.com/paper/1911.09723/) (2019)
- [PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning](https://mlsanity.com/paper/2001.00138/) (2020)
- [NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm](https://mlsanity.com/paper/1711.02017/) (2017)
- [ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1812.11677/) (2018)
- [Effective Model Sparsification by Scheduled Grow-and-Prune Methods](https://mlsanity.com/paper/2106.09857/) (2021)
- [Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)](https://mlsanity.com/paper/1807.06964/) (2018)
- [1xN Pattern for Pruning Convolutional Neural Networks](https://mlsanity.com/paper/2105.14713/) (2021)
- [Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08629/) (2021)
- [Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.03885/) (2022)
- [Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08819/) (2021)
- [Shfl-BW: Accelerating Deep Neural Network Inference with Tensor-Core Aware Weight Pruning](https://mlsanity.com/paper/2203.05016/) (2022)
- [SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration](https://mlsanity.com/paper/2310.06218/) (2023)

## Abstract (arXiv)

With the recent proliferation of on-device AI, there is an increasing need to run computationally intensive DNNs directly on mobile devices. However, the limited computing and memory resources of these devices necessitate effective pruning techniques. Block-wise pruning is promising due to its low accuracy drop tradeoff for speedup gains, but it requires block positions to be aligned with block size, hindering optimal position selection to minimize model accuracy drop. Unaligned block pruning (UBP) addresses this by allowing blocks to be selected at arbitrary positions, yet its practical use is limited by a time-consuming optimal block selection algorithm and lack of efficient inference kernels. In this paper, we propose a pseudo-optimal yet fast block selection algorithm called Block Expansion and Division (BED), which can be integrated into an iterative model training process. Additionally, we introduce an efficient inference kernel implementation for mobile devices, enabling a UBP-based model to achieve similar latency to a DNN model compressed by aligned block pruning. We demonstrate the superiority of our techniques on a real mobile phone with MobileNet and ResNet models.