# Extreme Network Compression via Filter Group Approximation

arXiv: https://arxiv.org/abs/1807.11254 · HTML: https://mlsanity.com/paper/1807.11254/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2018.

## Built on by (most cited first, 12 of 12)

- [Carrying out CNN Channel Pruning in a White Box](https://mlsanity.com/paper/2104.11883/) (2021)
- [Lottery Jackpots Exist in Pre-trained Models](https://mlsanity.com/paper/2104.08700/) (2021)
- [Model Preserving Compression for Neural Networks](https://mlsanity.com/paper/2108.00065/) (2021)
- [Update Compression for Deep Neural Networks on the Edge](https://mlsanity.com/paper/2203.04516/) (2022)
- [SVD-NAS: Coupling Low-Rank Approximation and Neural Architecture Search](https://mlsanity.com/paper/2208.10404/) (2022)
- [Learnable Heterogeneous Convolution: Learning both topology and strength](https://mlsanity.com/paper/2301.05440/) (2023)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [SMOF: Squeezing More Out of Filters Yields Hardware-Friendly CNN Pruning](https://mlsanity.com/paper/2110.10842/) (2021)
- [Infra-YOLO: Efficient Neural Network Structure with Model Compression for Real-Time Infrared Small Object Detection](https://mlsanity.com/paper/2408.07455/) (2024)
- [Differentiable Neural Architecture Learning for Efficient Neural Network Design](https://mlsanity.com/paper/2103.02126/) (2021)
- [Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints](https://mlsanity.com/paper/2108.01224/) (2021)
- [Lightweight Design and Optimization methods for DCNNs: Progress and Futures](https://mlsanity.com/paper/2412.16886/) (2024)

## Abstract (arXiv)

In this paper we propose a novel decomposition method based on filter group approximation, which can significantly reduce the redundancy of deep convolutional neural networks (CNNs) while maintaining the majority of feature representation. Unlike other low-rank decomposition algorithms which operate on spatial or channel dimension of filters, our proposed method mainly focuses on exploiting the filter group structure for each layer. For several commonly used CNN models, including VGG and ResNet, our method can reduce over 80% floating-point operations (FLOPs) with less accuracy drop than state-of-the-art methods on various image classification datasets. Besides, experiments demonstrate that our method is conducive to alleviating degeneracy of the compressed network, which hurts the convergence and performance of the network.