# Filter Sketch for Network Pruning

arXiv: https://arxiv.org/abs/2001.08514 · HTML: https://mlsanity.com/paper/2001.08514/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2020.

## Built on by (most cited first, 20 of 20)

- [Accelerating Sparse Deep Neural Networks](https://mlsanity.com/paper/2104.08378/) (2021)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers](https://mlsanity.com/paper/2402.04744/) (2024)
- [Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filters](https://mlsanity.com/paper/2202.07190/) (2022)
- [Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution](https://mlsanity.com/paper/2109.03075/) (2021)
- [Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask](https://mlsanity.com/paper/2209.07617/) (2022)
- [Quantifying Task Priority for Multi-Task Optimization](https://mlsanity.com/paper/2406.02996/) (2024)
- [Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN](https://mlsanity.com/paper/2107.14444/) (2021)
- [Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion](https://mlsanity.com/paper/2107.06916/) (2021)
- [Compact Model Training by Low-Rank Projection with Energy Transfer](https://mlsanity.com/paper/2204.05566/) (2022)
- [DyCE: Dynamically Configurable Exiting for Deep Learning Compression and Real-time Scaling](https://mlsanity.com/paper/2403.01695/) (2024)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Filter Pruning based on Information Capacity and Independence](https://mlsanity.com/paper/2303.03645/) (2023)
- [Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler](https://mlsanity.com/paper/2307.00198/) (2023)
- [An Improved Frequent Directions Algorithm for Low-Rank Approximation via Block Krylov Iteration](https://mlsanity.com/paper/2109.11703/) (2021)
- [Reconstructing Pruned Filters using Cheap Spatial Transformations](https://mlsanity.com/paper/2110.12844/) (2021)
- [A Multi-objective Complex Network Pruning Framework Based on Divide-and-conquer and Global Performance Impairment Ranking](https://mlsanity.com/paper/2303.16212/) (2023)
- [Beta-Rank: A Robust Convolutional Filter Pruning Method For Imbalanced Medical Image Analysis](https://mlsanity.com/paper/2304.07461/) (2023)
- [Reduced storage direct tensor ring decomposition for convolutional neural networks compression](https://mlsanity.com/paper/2405.10802/) (2024)
- [Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks](https://mlsanity.com/paper/2501.11135/) (2025)

## Abstract (arXiv)

We propose a novel network pruning approach by information preserving of pre-trained network weights (filters). Network pruning with the information preserving is formulated as a matrix sketch problem, which is efficiently solved by the off-the-shelf Frequent Direction method. Our approach, referred to as FilterSketch, encodes the second-order information of pre-trained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure. FilterSketch requires neither training from scratch nor data-driven iterative optimization, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3% of FLOPs and prunes 59.9% of network parameters with negligible accuracy cost for ResNet-110. On ILSVRC-2012, it reduces 45.5% of FLOPs and removes 43.0% of parameters with only 0.69% accuracy drop for ResNet-50. Our code and pruned models can be found at https://github.com/lmbxmu/FilterSketch.