# Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework

arXiv: https://arxiv.org/abs/2010.04879 · HTML: https://mlsanity.com/paper/2010.04879/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2020.

## Built on by (most cited first, 27 of 27)

- [DepGraph: Towards Any Structural Pruning](https://mlsanity.com/paper/2301.12900/) (2023)
- [Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2305.18010/) (2023)
- [Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning](https://mlsanity.com/paper/2210.01035/) (2022)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [Understanding the Dynamics of DNNs Using Graph Modularity](https://mlsanity.com/paper/2111.12485/) (2021)
- [Multi-Dimensional Model Compression of Vision Transformer](https://mlsanity.com/paper/2201.00043/) (2021)
- [Automatic Attention Pruning: Improving and Automating Model Pruning using Attentions](https://mlsanity.com/paper/2303.08595/) (2023)
- [CP3: Channel Pruning Plug-in for Point-based Networks](https://mlsanity.com/paper/2303.13097/) (2023)
- [Neuro-Symbolic Learning: Principles and Applications in Ophthalmology](https://mlsanity.com/paper/2208.00374/) (2022)
- [An Effective Information Theoretic Framework for Channel Pruning](https://mlsanity.com/paper/2408.16772/) (2024)
- [Dynamic Structure Pruning for Compressing CNNs](https://mlsanity.com/paper/2303.09736/) (2023)
- [Boosting Pruned Networks with Linear Over-parameterization](https://mlsanity.com/paper/2204.11444/) (2022)
- [Slimmable Networks for Contrastive Self-supervised Learning](https://mlsanity.com/paper/2209.15525/) (2022)
- [RedTest: Towards Measuring Redundancy in Deep Neural Networks Effectively](https://mlsanity.com/paper/2411.10507/) (2024)
- [From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks](https://mlsanity.com/paper/2405.06038/) (2024)
- [Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training](https://mlsanity.com/paper/2302.10798/) (2023)
- [On Model Explanations with Transferable Neural Pathways](https://mlsanity.com/paper/2309.09887/) (2023)
- [Complexity-Aware Training of Deep Neural Networks for Optimal Structure Discovery](https://mlsanity.com/paper/2411.09127/) (2024)
- [Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices](https://mlsanity.com/paper/2110.08013/) (2021)
- [Temporally Resolution Decrement: Utilizing the Shape Consistency for Higher Computational Efficiency](https://mlsanity.com/paper/2112.00954/) (2021)
- [A One-Shot Reparameterization Method for Reducing the Loss of Tile Pruning on DNNs](https://mlsanity.com/paper/2207.14545/) (2022)
- [CWP: Instance complexity weighted channel-wise soft masks for network pruning](https://mlsanity.com/paper/2209.03534/) (2022)
- [Surrogate Lagrangian Relaxation: A Path To Retrain-free Deep Neural Network Pruning](https://mlsanity.com/paper/2304.04120/) (2023)
- [Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications](https://mlsanity.com/paper/2402.07229/) (2024)
- [A Comprehensive Study of Structural Pruning for Vision Models](https://mlsanity.com/paper/2406.12315/) (2024)
- [Hyperbolic Binary Neural Network](https://mlsanity.com/paper/2501.03471/) (2025)
- [C-SWAP: Explainability-Aware Structured Pruning for Efficient Neural Networks Compression](https://mlsanity.com/paper/2510.18636/) (2025)

## Abstract (arXiv)

Most neural network pruning methods, such as filter-level and layer-level prunings, prune the network model along one dimension (depth, width, or resolution) solely to meet a computational budget. However, such a pruning policy often leads to excessive reduction of that dimension, thus inducing a huge accuracy loss. To alleviate this issue, we argue that pruning should be conducted along three dimensions comprehensively. For this purpose, our pruning framework formulates pruning as an optimization problem. Specifically, it first casts the relationships between a certain model's accuracy and depth/width/resolution into a polynomial regression and then maximizes the polynomial to acquire the optimal values for the three dimensions. Finally, the model is pruned along the three optimal dimensions accordingly. In this framework, since collecting too much data for training the regression is very time-costly, we propose two approaches to lower the cost: 1) specializing the polynomial to ensure an accurate regression even with less training data; 2) employing iterative pruning and fine-tuning to collect the data faster. Extensive experiments show that our proposed algorithm surpasses state-of-the-art pruning algorithms and even neural architecture search-based algorithms.