# AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

arXiv: https://arxiv.org/abs/1907.03141 · HTML: https://mlsanity.com/paper/1907.03141/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2019.

## Built on by (most cited first, 30 of 49)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning](https://mlsanity.com/paper/2102.03214/) (2021)
- [SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning](https://mlsanity.com/paper/2110.11395/) (2021)
- [Follow Anything: Open-set detection, tracking, and following in real-time](https://mlsanity.com/paper/2308.05737/) (2023)
- [Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?](https://mlsanity.com/paper/2102.11068/) (2021)
- [Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm](https://mlsanity.com/paper/2110.08190/) (2021)
- [Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference](https://mlsanity.com/paper/2301.09254/) (2023)
- [Stealthy Backdoors as Compression Artifacts](https://mlsanity.com/paper/2104.15129/) (2021)
- [TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge](https://mlsanity.com/paper/2307.09988/) (2023)
- [Differentiable Network Pruning for Microcontrollers](https://mlsanity.com/paper/2110.08350/) (2021)
- [Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filters](https://mlsanity.com/paper/2202.07190/) (2022)
- [An Automatic and Efficient BERT Pruning for Edge AI Systems](https://mlsanity.com/paper/2206.10461/) (2022)
- [APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Cores](https://mlsanity.com/paper/2106.12169/) (2021)
- [LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms](https://mlsanity.com/paper/2311.11420/) (2023)
- [GOHSP: A Unified Framework of Graph and Optimization-based Heterogeneous Structured Pruning for Vision Transformer](https://mlsanity.com/paper/2301.05345/) (2023)
- [Performance-aware Approximation of Global Channel Pruning for Multitask CNNs](https://mlsanity.com/paper/2303.11923/) (2023)
- [Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression](https://mlsanity.com/paper/2107.12445/) (2021)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models](https://mlsanity.com/paper/2503.15369/) (2025)
- [CATRO: Channel Pruning via Class-Aware Trace Ratio Optimization](https://mlsanity.com/paper/2110.10921/) (2021)
- [EPSD: Early Pruning with Self-Distillation for Efficient Model Compression](https://mlsanity.com/paper/2402.00084/) (2024)
- [All-in-One: A Highly Representative DNN Pruning Framework for Edge Devices with Dynamic Power Management](https://mlsanity.com/paper/2212.05122/) (2022)
- [Temporal Action Detection Model Compression by Progressive Block Drop](https://mlsanity.com/paper/2503.16916/) (2025)
- [Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization](https://mlsanity.com/paper/2101.04073/) (2021)
- [ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search](https://mlsanity.com/paper/2110.03858/) (2021)
- [YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers](https://mlsanity.com/paper/2203.03794/) (2022)
- [Towards Hardware-Specific Automatic Compression of Neural Networks](https://mlsanity.com/paper/2212.07818/) (2022)
- [Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes](https://mlsanity.com/paper/2405.05808/) (2024)

## Abstract (arXiv)

Structured weight pruning is a representative model compression technique of DNNs to reduce the storage and computation requirements and accelerate inference. An automatic hyperparameter determination process is necessary due to the large number of flexible hyperparameters. This work proposes AutoCompress, an automatic structured pruning framework with the following key performance improvements: (i) effectively incorporate the combination of structured pruning schemes in the automatic process; (ii) adopt the state-of-art ADMM-based structured weight pruning as the core algorithm, and propose an innovative additional purification step for further weight reduction without accuracy loss; and (iii) develop effective heuristic search method enhanced by experience-based guided search, replacing the prior deep reinforcement learning technique which has underlying incompatibility with the target pruning problem. Extensive experiments on CIFAR-10 and ImageNet datasets demonstrate that AutoCompress is the key to achieve ultra-high pruning rates on the number of weights and FLOPs that cannot be achieved before. As an example, AutoCompress outperforms the prior work on automatic model compression by up to 33x in pruning rate (120x reduction in the actual parameter count) under the same accuracy. Significant inference speedup has been observed from the AutoCompress framework on actual measurements on smartphone. We release all models of this work at anonymous link: http://bit.ly/2VZ63dS.