# Learning Filter Basis for Convolutional Neural Network Compression

arXiv: https://arxiv.org/abs/1908.08932 · HTML: https://mlsanity.com/paper/1908.08932/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2019.

## Built on by (most cited first, 30 of 57)

- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [Blueprint Separable Residual Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2205.05996/) (2022)
- [NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results](https://mlsanity.com/paper/2205.05675/) (2022)
- [Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07809/) (2021)
- [Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07825/) (2021)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Bit-Mixer: Mixed-precision networks with runtime bit-width selection](https://mlsanity.com/paper/2103.17267/) (2021)
- [Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions](https://mlsanity.com/paper/2209.08554/) (2022)
- [Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08630/) (2021)
- [Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08826/) (2021)
- [Towards Efficient Graph Convolutional Networks for Point Cloud Handling](https://mlsanity.com/paper/2104.05706/) (2021)
- [A Unified Approach to Coreset Learning](https://mlsanity.com/paper/2111.03044/) (2021)
- [Edge-enhanced Feature Distillation Network for Efficient Super-Resolution](https://mlsanity.com/paper/2204.08759/) (2022)
- [Blending Anti-Aliasing into Vision Transformer](https://mlsanity.com/paper/2110.15156/) (2021)
- [Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs](https://mlsanity.com/paper/2203.07808/) (2022)
- [Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05910/) (2022)
- [Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08629/) (2021)
- [Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners](https://mlsanity.com/paper/2212.08066/) (2022)
- [Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.03885/) (2022)
- [Geometry-aware training of factorized layers in tensor Tucker format](https://mlsanity.com/paper/2305.19059/) (2023)
- [Provable Emergence of Deep Neural Collapse and Low-Rank Bias in L2-Regularized Nonlinear Networks](https://mlsanity.com/paper/2402.03991/) (2024)
- [Large Convolutional Model Tuning via Filter Subspace](https://mlsanity.com/paper/2403.00269/) (2024)
- [Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08819/) (2021)
- [Deep Model Compression based on the Training History](https://mlsanity.com/paper/2102.00160/) (2021)
- [Less is More: Accelerating Faster Neural Networks Straight from JPEG](https://mlsanity.com/paper/2104.00185/) (2021)
- [TT-NF: Tensor Train Neural Fields](https://mlsanity.com/paper/2209.15529/) (2022)
- [Toward Compact Deep Neural Networks via Energy-Aware Pruning](https://mlsanity.com/paper/2103.10858/) (2021)
- [Frequency learning for structured CNN filters with Gaussian fractional derivatives](https://mlsanity.com/paper/2111.06660/) (2021)

## Abstract (arXiv)

Convolutional neural networks (CNNs) based solutions have achieved state-of-the-art performances for many computer vision tasks, including classification and super-resolution of images. Usually the success of these methods comes with a cost of millions of parameters due to stacking deep convolutional layers. Moreover, quite a large number of filters are also used for a single convolutional layer, which exaggerates the parameter burden of current methods. Thus, in this paper, we try to reduce the number of parameters of CNNs by learning a basis of the filters in convolutional layers. For the forward pass, the learned basis is used to approximate the original filters and then used as parameters for the convolutional layers. We validate our proposed solution for multiple CNN architectures on image classification and image super-resolution benchmarks and compare favorably to the existing state-of-the-art in terms of reduction of parameters and preservation of accuracy.