# Attention-Based Guided Structured Sparsity of Deep Neural Networks

arXiv: https://arxiv.org/abs/1802.09902 · HTML: https://mlsanity.com/paper/1802.09902/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2018.

## Built on by (most cited first, 2 of 2)

- [E2-AEN: End-to-End Incremental Learning with Adaptively Expandable Network](https://mlsanity.com/paper/2207.06754/) (2022)
- [Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation](https://mlsanity.com/paper/2607.23181/) (2026)

## Abstract (arXiv)

Network pruning is aimed at imposing sparsity in a neural network architecture by increasing the portion of zero-valued weights for reducing its size regarding energy-efficiency consideration and increasing evaluation speed. In most of the conducted research efforts, the sparsity is enforced for network pruning without any attention to the internal network characteristics such as unbalanced outputs of the neurons or more specifically the distribution of the weights and outputs of the neurons. That may cause severe accuracy drop due to uncontrolled sparsity. In this work, we propose an attention mechanism that simultaneously controls the sparsity intensity and supervised network pruning by keeping important information bottlenecks of the network to be active. On CIFAR-10, the proposed method outperforms the best baseline method by 6% and reduced the accuracy drop by 2.6x at the same level of sparsity.