Data to 7 Oct 2026

Learning Efficient Convolutional Networks through Network Slimming

arXiv 1708.06519 ↗First posted 22 Aug 2017cs.CV cs.AI cs.LGcited by 673

Not on the map. Why?

Abstract (arXiv)

The deployment of deep convolutional neural networks (CNNs) in many real world applications is largely hindered by their high computational cost. In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy. This is achieved by enforcing channel-level sparsity in the network in a simple but effective way. Different from many existing approaches, the proposed method directly applies to modern CNN architectures, introduces minimum overhead to the training process, and requires no special software/hardware accelerators for the resulting models. We call our approach network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. We empirically demonstrate the effectiveness of our approach with several state-of-the-art CNN models, including VGGNet, ResNet and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming gives a 20x reduction in model size and a 5x reduction in computing operations.

Key facts. Cited by 673 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 156; 2022: 142; 2023: 137; 2024: 96; 2025: 81; 2026: 61.

673 papers.

Sort:

By area:

2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
2021Dynamic Neural Networks: A Surveycited by 405
2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 283
2021Post-Training Quantization for Vision Transformercited by 274
2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
2023DepGraph: Towards Any Structural Pruningcited by 202
2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
2023Structural Pruning for Diffusion Modelscited by 193

38 more

2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
2022Structured Pruning Learns Compact and Accurate Modelscited by 167
2022On-Device Training Under 256KB Memorycited by 158
2021Patch Slimming for Efficient Vision Transformerscited by 131
2021Group Fisher Pruning for Practical Network Compressioncited by 130
2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
2021Dynamic Slimmable Networkcited by 83
2022Unified Visual Transformer Compressioncited by 79
2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
2021Vision Transformer Pruningcited by 77
2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifierscited by 66
2024LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insightscited by 65
2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
2023Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantizationcited by 59
2022Advancing Model Pruning via Bi-level Optimizationcited by 55
2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
2021Self-Damaging Contrastive Learningcited by 48
2021Differentiable Subset Pruning of Transformer Headscited by 46
2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
2023EVC: Towards Real-Time Neural Image Compression with Mask Decaycited by 46
2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 45
2022Learning to Weight Samples for Dynamic Early-exiting Networkscited by 44
2022Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Modelscited by 44
2021Hessian-Aware Pruning and Optimal Neural Implantcited by 43
2021Manifold Regularized Dynamic Network Pruningcited by 43
2024SlimGPT: Layer-wise Structured Pruning for Large Language Modelscited by 43
2022Membership Inference Attacks and Defenses in Neural Network Pruningcited by 42
2022EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Predictioncited by 42
2023The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Mattercited by 42
2021Recent Advances on Neural Network Pruning at Initializationcited by 41
2022Glance and Focus Networks for Dynamic Visual Recognitioncited by 41
2022Sparsity Winning Twice: Better Robust Generalization from More Efficient Trainingcited by 41

50 most cited of 673 All 673 (JSON)