# Cascaded Projection: End-to-End Network Compression and Acceleration

arXiv: https://arxiv.org/abs/1903.04988 · HTML: https://mlsanity.com/paper/1903.04988/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks](https://mlsanity.com/paper/2401.10341/) (2024)
- [Anytime-Lidar: Deadline-aware 3D Object Detection](https://mlsanity.com/paper/2208.12181/) (2022)
- [Grassmann Iterative Linear Discriminant Analysis with Proxy Matrix Optimization](https://mlsanity.com/paper/2104.08112/) (2021)
- [Compressing Deep CNNs using Basis Representation and Spectral Fine-tuning](https://mlsanity.com/paper/2105.10436/) (2021)

## Abstract (arXiv)

We propose a data-driven approach for deep convolutional neural network compression that achieves high accuracy with high throughput and low memory requirements. Current network compression methods either find a low-rank factorization of the features that requires more memory, or select only a subset of features by pruning entire filter channels. We propose the Cascaded Projection (CaP) compression method that projects the output and input filter channels of successive layers to a unified low dimensional space based on a low-rank projection. We optimize the projection to minimize classification loss and the difference between the next layer's features in the compressed and uncompressed networks. To solve this non-convex optimization problem we propose a new optimization method of a proxy matrix using backpropagation and Stochastic Gradient Descent (SGD) with geometric constraints. Our cascaded projection approach leads to improvements in all critical areas of network compression: high accuracy, low memory consumption, low parameter count and high processing speed. The proposed CaP method demonstrates state-of-the-art results compressing VGG16 and ResNet networks with over 4x reduction in the number of computations and excellent performance in top-5 accuracy on the ImageNet dataset before and after fine-tuning.