# Channel Pruning In Quantization-aware Training: An Adaptive Projection-gradient Descent-shrinkage-splitting Method

arXiv: https://arxiv.org/abs/2204.04375 · HTML: https://mlsanity.com/paper/2204.04375/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 17 references that are in the map.
- First posted on arXiv 9 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Adversarial Robustness vs Model Compression, or Both?](https://mlsanity.com/paper/1903.12561/) (2019)
- [Training Neural Networks Without Gradients: A Scalable ADMM Approach](https://mlsanity.com/paper/1605.02026/) (2016)
- [BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights](https://mlsanity.com/paper/1801.06313/) (2018)
- [Demystifying and Generalizing BinaryConnect](https://mlsanity.com/paper/2110.13220/) (2021)
- [Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets](https://mlsanity.com/paper/2003.00631/) (2020)
- [An Integrated Approach to Produce Robust Models with High Efficiency](https://mlsanity.com/paper/2008.13305/) (2020)
- [Convergence of a Relaxed Variable Splitting Method for Learning Sparse Neural Networks via ℓ1, ℓ0, and transformed-ℓ1 Penalties](https://mlsanity.com/paper/1812.05719/) (2018)
- [A Channel-Pruned and Weight-Binarized Convolutional Neural Network for Keyword Spotting](https://mlsanity.com/paper/1909.05623/) (2019)

## Abstract (arXiv)

We propose an adaptive projection-gradient descent-shrinkage-splitting method (APGDSSM) to integrate penalty based channel pruning into quantization-aware training (QAT). APGDSSM concurrently searches weights in both the quantized subspace and the sparse subspace. APGDSSM uses shrinkage operator and a splitting technique to create sparse weights, as well as the Group Lasso penalty to push the weight sparsity into channel sparsity. In addition, we propose a novel complementary transformed l1 penalty to stabilize the training for extreme compression.