# Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression

arXiv: https://arxiv.org/abs/1803.05729 · HTML: https://mlsanity.com/paper/1803.05729/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2018.

## Built on by (most cited first, 11 of 11)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation](https://mlsanity.com/paper/2307.12574/) (2023)
- [Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN](https://mlsanity.com/paper/2107.14444/) (2021)
- [Survey: Exploiting Data Redundancy for Optimization of Deep Learning](https://mlsanity.com/paper/2208.13363/) (2022)
- [Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning](https://mlsanity.com/paper/2209.13590/) (2022)
- [Homological Neural Networks: A Sparse Architecture for Multivariate Complexity](https://mlsanity.com/paper/2306.15337/) (2023)
- [Receding Neuron Importances for Structured Pruning](https://mlsanity.com/paper/2204.06404/) (2022)
- [Filter Pruning based on Information Capacity and Independence](https://mlsanity.com/paper/2303.03645/) (2023)
- [Compressing Deep CNNs using Basis Representation and Spectral Fine-tuning](https://mlsanity.com/paper/2105.10436/) (2021)
- [Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning](https://mlsanity.com/paper/2206.03596/) (2022)
- [Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis](https://mlsanity.com/paper/2604.23375/) (2026)

## Abstract (arXiv)

While the research on convolutional neural networks (CNNs) is progressing quickly, the real-world deployment of these models is often limited by computing resources and memory constraints. In this paper, we address this issue by proposing a novel filter pruning method to compress and accelerate CNNs. Our work is based on the linear relationship identified in different feature map subspaces via visualization of feature maps. Such linear relationship implies that the information in CNNs is redundant. Our method eliminates the redundancy in convolutional filters by applying subspace clustering to feature maps. In this way, most of the representative information in the network can be retained in each cluster. Therefore, our method provides an effective solution to filter pruning for which most existing methods directly remove filters based on simple heuristics. The proposed method is independent of the network structure, thus it can be adopted by any off-the-shelf deep learning libraries. Experiments on different networks and tasks show that our method outperforms existing techniques before fine-tuning, and achieves the state-of-the-art results after fine-tuning.