# Efficient CNN Compression via Multi-method Low Rank Factorization and Feature Map Similarity

arXiv: https://arxiv.org/abs/2510.00062 · HTML: https://mlsanity.com/paper/2510.00062/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [TensorLy: Tensor Learning in Python](https://mlsanity.com/paper/1610.09555/) (2016)
- [Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network](https://mlsanity.com/paper/2008.05441/) (2020)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compression](https://mlsanity.com/paper/1701.07148/) (2017)
- [HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks](https://mlsanity.com/paper/2301.09422/) (2023)
- [ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks](https://mlsanity.com/paper/2401.10341/) (2024)
- [Joint Matrix Decomposition for Deep Convolutional Neural Networks Compression](https://mlsanity.com/paper/2107.04386/) (2021)
- [Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition](https://mlsanity.com/paper/2309.12412/) (2023)
- [Rank Selection of CP-decomposed Convolutional Layers with Variational Bayesian Matrix Factorization](https://mlsanity.com/paper/1801.05243/) (2018)
- [Canonical Polyadic Decomposition and Deep Learning for Machine Fault Detection](https://mlsanity.com/paper/2107.09519/) (2021)
- [InDistill: Information flow-preserving knowledge distillation for model compression](https://mlsanity.com/paper/2205.10003/) (2022)
- [How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?](https://mlsanity.com/paper/2305.05318/) (2023)
- [Filter-Pruning of Lightweight Face Detectors Using a Geometric Median Criterion](https://mlsanity.com/paper/2311.16613/) (2023)

## Abstract (arXiv)

Low-Rank Factorization (LRF) is a widely adopted technique for compressing deep neural networks (DNNs). However, it faces several challenges, including optimal rank selection, a vast design space, long fine-tuning times, and limited compatibility with different layer types and decomposition methods. This paper presents an end-to-end Design Space Exploration (DSE) methodology and framework for compressing convolutional neural networks (CNNs) that addresses all these issues. We introduce a novel rank selection strategy based on feature map similarity, which captures non-linear interactions between layer outputs more effectively than traditional weight-based approaches. Unlike prior works, our method uses a one-shot fine-tuning process, significantly reducing the overall fine-tuning time. The proposed framework is fully compatible with all types of convolutional (Conv) and fully connected (FC) layers. To further improve compression, the framework integrates three different LRF techniques for Conv layers and three for FC layers, applying them selectively on a per-layer basis. We demonstrate that combining multiple LRF methods within a single model yields better compression results than using a single method uniformly across all layers. Finally, we provide a comprehensive evaluation and comparison of the six LRF techniques, offering practical insights into their effectiveness across different scenarios. The proposed work is integrated into TensorFlow 2.x, ensuring compatibility with widely used deep learning workflows. Experimental results on 14 CNN models across eight datasets demonstrate that the proposed methodology achieves substantial compression with minimal accuracy loss, outperforming several state-of-the-art techniques.