# PCAS: Pruning Channels with Attention Statistics for Deep Network Compression

arXiv: https://arxiv.org/abs/1806.05382 · HTML: https://mlsanity.com/paper/1806.05382/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2018.

## Built on by (most cited first, 7 of 7)

- [Carrying out CNN Channel Pruning in a White Box](https://mlsanity.com/paper/2104.11883/) (2021)
- [Complexity-Driven CNN Compression for Resource-constrained Edge AI](https://mlsanity.com/paper/2208.12816/) (2022)
- [UPSCALE: Unconstrained Channel Pruning](https://mlsanity.com/paper/2307.08771/) (2023)
- [Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approach](https://mlsanity.com/paper/2111.02399/) (2021)
- [Gator: Customizable Channel Pruning of Neural Networks with Gating](https://mlsanity.com/paper/2205.15404/) (2022)
- [Rewarded meta-pruning: Meta Learning with Rewards for Channel Pruning](https://mlsanity.com/paper/2301.11063/) (2023)
- [Compressive Feature Selection for Remote Visual Multi-Task Inference](https://mlsanity.com/paper/2405.09077/) (2024)

## Abstract (arXiv)

Compression techniques for deep neural networks are important for implementing them on small embedded devices. In particular, channel-pruning is a useful technique for realizing compact networks. However, many conventional methods require manual setting of compression ratios in each layer. It is difficult to analyze the relationships between all layers, especially for deeper models. To address these issues, we propose a simple channel-pruning technique based on attention statistics that enables to evaluate the importance of channels. We improved the method by means of a criterion for automatic channel selection, using a single compression ratio for the entire model in place of per-layer model analysis. The proposed approach achieved superior performance over conventional methods with respect to accuracy and the computational costs for various models and datasets. We provide analysis results for behavior of the proposed criterion on different datasets to demonstrate its favorable properties for channel pruning.