# Group channel pruning and spatial attention distilling for object detection

arXiv: https://arxiv.org/abs/2306.01526 · HTML: https://mlsanity.com/paper/2306.01526/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 61 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://mlsanity.com/paper/1707.06342/) (2017)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning](https://mlsanity.com/paper/2007.02491/) (2020)
- [AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference](https://mlsanity.com/paper/1805.08941/) (2018)
- [Pelee: A Real-Time Object Detection System on Mobile Devices](https://mlsanity.com/paper/1804.06882/) (2018)
- [STDP Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy Efficient Recognition](https://mlsanity.com/paper/1710.04734/) (2017)
- [ThunderNet: Towards Real-time Generic Object Detection](https://mlsanity.com/paper/1903.11752/) (2019)
- [Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey](https://mlsanity.com/paper/2001.04074/) (2020)
- [Search to Distill: Pearls are Everywhere but not the Eyes](https://mlsanity.com/paper/1911.09074/) (2019)
- [OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization](https://mlsanity.com/paper/2205.11141/) (2022)
- [Tiny-DSOD: Lightweight Object Detection for Resource-Restricted Usages](https://mlsanity.com/paper/1807.11013/) (2018)
- [DC-SPP-YOLO: Dense Connection and Spatial Pyramid Pooling Based YOLO for Object Detection](https://mlsanity.com/paper/1903.08589/) (2019)
- [Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligence](https://mlsanity.com/paper/1910.01010/) (2019)
- [Edge Network-Assisted Real-Time Object Detection Framework for Autonomous Driving](https://mlsanity.com/paper/2008.07083/) (2020)

## Built on by (most cited first, 1 of 1)

- [Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review](https://mlsanity.com/paper/2508.03317/) (2025)

## Abstract (arXiv)

Due to the over-parameterization of neural networks, many model compression methods based on pruning and quantization have emerged. They are remarkable in reducing the size, parameter number, and computational complexity of the model. However, most of the models compressed by such methods need the support of special hardware and software, which increases the deployment cost. Moreover, these methods are mainly used in classification tasks, and rarely directly used in detection tasks. To address these issues, for the object detection network we introduce a three-stage model compression method: dynamic sparse training, group channel pruning, and spatial attention distilling. Firstly, to select out the unimportant channels in the network and maintain a good balance between sparsity and accuracy, we put forward a dynamic sparse training method, which introduces a variable sparse rate, and the sparse rate will change with the training process of the network. Secondly, to reduce the effect of pruning on network accuracy, we propose a novel pruning method called group channel pruning. In particular, we divide the network into multiple groups according to the scales of the feature layer and the similarity of module structure in the network, and then we use different pruning thresholds to prune the channels in each group. Finally, to recover the accuracy of the pruned network, we use an improved knowledge distillation method for the pruned network. Especially, we extract spatial attention information from the feature maps of specific scales in each group as knowledge for distillation. In the experiments, we use YOLOv4 as the object detection network and PASCAL VOC as the training dataset. Our method reduces the parameters of the model by 64.7 % and the calculation by 34.9%.