# A Low Effort Approach to Structured CNN Design Using PCA

arXiv: https://arxiv.org/abs/1812.06224 · HTML: https://mlsanity.com/paper/1812.06224/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2018.

## Built on by (most cited first, 14 of 14)

- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networks](https://mlsanity.com/paper/2104.12528/) (2021)
- [coVariance Neural Networks](https://mlsanity.com/paper/2205.15856/) (2022)
- [Mixed-Precision Neural Networks: A Survey](https://mlsanity.com/paper/2208.06064/) (2022)
- [rm SP3: Enhancing Structured Pruning via PCA Projection](https://mlsanity.com/paper/2308.16475/) (2023)
- [Revealing the Utilized Rank of Subspaces of Learning in Neural Networks](https://mlsanity.com/paper/2407.04797/) (2024)
- [Complexity-aware Adaptive Training and Inference for Edge-Cloud Distributed AI Systems](https://mlsanity.com/paper/2109.06440/) (2021)
- [Spatio-Temporal Pruning for Compressed Spiking Large Language Models](https://mlsanity.com/paper/2508.20122/) (2025)
- [Basis Scaling and Double Pruning for Efficient Inference in Network-Based Transfer Learning](https://mlsanity.com/paper/2108.02893/) (2021)
- [An Acceleration Method Based on Deep Learning and Multilinear Feature Space](https://mlsanity.com/paper/2110.08679/) (2021)
- [Pruning for Improved ADC Efficiency in Crossbar-based Analog In-memory Accelerators](https://mlsanity.com/paper/2403.13082/) (2024)
- [Towards Generalized Entropic Sparsification for Convolutional Neural Networks](https://mlsanity.com/paper/2404.04734/) (2024)
- [Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression](https://mlsanity.com/paper/2407.08744/) (2024)
- [SPEAR: Structured Pruning for Spiking Neural Networks via Synaptic Operation Estimation and Reinforcement Learning](https://mlsanity.com/paper/2507.02945/) (2025)

## Abstract (arXiv)

Deep learning models hold state of the art performance in many fields, yet their design is still based on heuristics or grid search methods that often result in overparametrized networks. This work proposes a method to analyze a trained network and deduce an optimized, compressed architecture that preserves accuracy while keeping computational costs tractable. Model compression is an active field of research that targets the problem of realizing deep learning models in hardware. However, most pruning methodologies tend to be experimental, requiring large compute and time intensive iterations of retraining the entire network. We introduce structure into model design by proposing a single shot analysis of a trained network that serves as a first order, low effort approach to dimensionality reduction, by using PCA (Principal Component Analysis). The proposed method simultaneously analyzes the activations of each layer and considers the dimensionality of the space described by the filters generating these activations. It optimizes the architecture in terms of number of layers, and number of filters per layer without any iterative retraining procedures, making it a viable, low effort technique to design efficient networks. We demonstrate the proposed methodology on AlexNet and VGG style networks on the CIFAR-10, CIFAR-100 and ImageNet datasets, and successfully achieve an optimized architecture with a reduction of up to 3.8X and 9X in the number of operations and parameters respectively, while trading off less than 1% accuracy. We also apply the method to MobileNet, and achieve 1.7X and 3.9X reduction in the number of operations and parameters respectively, while improving accuracy by almost one percentage point.