# Training CNNs with Low-Rank Filters for Efficient Image Classification

arXiv: https://arxiv.org/abs/1511.06744 · HTML: https://mlsanity.com/paper/1511.06744/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2015.

## Built on by (most cited first, 25 of 25)

- [ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models](https://mlsanity.com/paper/2312.05821/) (2023)
- [Pufferfish: Communication-efficient Models At No Extra Cost](https://mlsanity.com/paper/2103.03936/) (2021)
- [Initialization and Regularization of Factorized Neural Layers](https://mlsanity.com/paper/2105.01029/) (2021)
- [Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations](https://mlsanity.com/paper/2205.13571/) (2022)
- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [projUNN: efficient method for training deep networks with unitary matrices](https://mlsanity.com/paper/2203.05483/) (2022)
- [ResFields: Residual Neural Fields for Spatiotemporal Signals](https://mlsanity.com/paper/2309.03160/) (2023)
- [An Information-Theoretic Justification for Model Pruning](https://mlsanity.com/paper/2102.08329/) (2021)
- [Cuttlefish: Low-Rank Model Training without All the Tuning](https://mlsanity.com/paper/2305.02538/) (2023)
- [Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions](https://mlsanity.com/paper/2209.08554/) (2022)
- [Federated Learning of Large Models at the Edge via Principal Sub-Model Training](https://mlsanity.com/paper/2208.13141/) (2022)
- [Geometry-aware training of factorized layers in tensor Tucker format](https://mlsanity.com/paper/2305.19059/) (2023)
- [ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks](https://mlsanity.com/paper/2401.10341/) (2024)
- [InRank: Incremental Low-Rank Learning](https://mlsanity.com/paper/2306.11250/) (2023)
- [ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2210.05280/) (2022)
- [All Rivers Run to the Sea: Private Learning with Asymmetric Flows](https://mlsanity.com/paper/2312.05264/) (2023)
- [TropNNC: Structured Neural Network Compression Using Tropical Geometry](https://mlsanity.com/paper/2409.03945/) (2024)
- [Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models](https://mlsanity.com/paper/2410.07771/) (2024)
- [Filter Pruning based on Information Capacity and Independence](https://mlsanity.com/paper/2303.03645/) (2023)
- [Low-Rank Compression for IMC Arrays](https://mlsanity.com/paper/2502.07820/) (2025)
- [Edge Private Graph Neural Networks with Singular Value Perturbation](https://mlsanity.com/paper/2403.10995/) (2024)
- [Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples](https://mlsanity.com/paper/2510.20800/) (2025)
- [Optimization of Layer Skipping and Frequency Scaling for Convolutional Neural Networks under Latency Constraint](https://mlsanity.com/paper/2503.24014/) (2025)
- [Hierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis](https://mlsanity.com/paper/2604.23375/) (2026)

## Abstract (arXiv)

We propose a new method for creating computationally efficient convolutional neural networks (CNNs) by using low-rank representations of convolutional filters. Rather than approximating filters in previously-trained networks with more efficient versions, we learn a set of small basis filters from scratch; during training, the network learns to combine these basis filters into more complex filters that are discriminative for image classification. To train such networks, a novel weight initialization scheme is used. This allows effective initialization of connection weights in convolutional layers composed of groups of differently-shaped filters. We validate our approach by applying it to several existing CNN architectures and training these networks from scratch using the CIFAR, ILSVRC and MIT Places datasets. Our results show similar or higher accuracy than conventional CNNs with much less compute. Applying our method to an improved version of VGG-11 network using global max-pooling, we achieve comparable validation accuracy using 41% less compute and only 24% of the original VGG-11 model parameters; another variant of our method gives a 1 percentage point increase in accuracy over our improved VGG-11 model, giving a top-5 center-crop validation accuracy of 89.7% while reducing computation by 16% relative to the original VGG-11 model. Applying our method to the GoogLeNet architecture for ILSVRC, we achieved comparable accuracy with 26% less compute and 41% fewer model parameters. Applying our method to a near state-of-the-art network for CIFAR, we achieved comparable accuracy with 46% less compute and 55% fewer parameters.