# MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning

arXiv: https://arxiv.org/abs/1903.10258 · HTML: https://mlsanity.com/paper/1903.10258/

## Key facts

- Cited by 227 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2019.

## Built on by (most cited first, 30 of 227)

- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [DepGraph: Towards Any Structural Pruning](https://mlsanity.com/paper/2301.12900/) (2023)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [On-Device Training Under 256KB Memory](https://mlsanity.com/paper/2206.15472/) (2022)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [Group Fisher Pruning for Practical Network Compression](https://mlsanity.com/paper/2108.00708/) (2021)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning](https://mlsanity.com/paper/2110.15352/) (2021)
- [Dynamic Slimmable Network](https://mlsanity.com/paper/2103.13258/) (2021)
- [Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation](https://mlsanity.com/paper/2111.14826/) (2021)
- [Oscillation-free Quantization for Low-bit Vision Transformers](https://mlsanity.com/paper/2302.02210/) (2023)
- [The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report](https://mlsanity.com/paper/2404.10343/) (2024)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [Meta-Learning Sparse Implicit Neural Representations](https://mlsanity.com/paper/2110.14678/) (2021)
- [Hessian-Aware Pruning and Optimal Neural Implant](https://mlsanity.com/paper/2101.08940/) (2021)
- [Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models](https://mlsanity.com/paper/2211.02048/) (2022)
- [Manifold Regularized Dynamic Network Pruning](https://mlsanity.com/paper/2103.05861/) (2021)
- [Structural Pruning via Latency-Saliency Knapsack](https://mlsanity.com/paper/2210.06659/) (2022)
- [NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results](https://mlsanity.com/paper/2205.05675/) (2022)
- [Tiny Machine Learning: Progress and Futures](https://mlsanity.com/paper/2403.19076/) (2024)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [CHEX: CHannel EXploration for CNN Model Compression](https://mlsanity.com/paper/2203.15794/) (2022)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07809/) (2021)
- [Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07825/) (2021)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection](https://mlsanity.com/paper/2205.11098/) (2022)

## Abstract (arXiv)

In this paper, we propose a novel meta learning approach for automatic channel pruning of very deep neural networks. We first train a PruningNet, a kind of meta network, which is able to generate weight parameters for any pruned structure given the target network. We use a simple stochastic structure sampling method for training the PruningNet. Then, we apply an evolutionary procedure to search for good-performing pruned networks. The search is highly efficient because the weights are directly generated by the trained PruningNet and we do not need any finetuning at search time. With a single PruningNet trained for the target network, we can search for various Pruned Networks under different constraints with little human participation. Compared to the state-of-the-art pruning methods, we have demonstrated superior performances on MobileNet V1/V2 and ResNet. Codes are available on https://github.com/liuzechun/MetaPruning.