mlsanityData to 7 Oct 2026

Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition

arXiv 2109.14710 ↗First posted 29 Sep 2021cs.CV cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Modern Convolutional Neural Network (CNN) architectures, despite their superiority in solving various problems, are generally too large to be deployed on resource constrained edge devices. In this paper, we reduce memory usage and floating-point operations required by convolutional layers in CNNs. We compress these layers by generalizing the Kronecker Product Decomposition to apply to multidimensional tensors, leading to the Generalized Kronecker Product Decomposition (GKPD). Our approach yields a plug-and-play module that can be used as a drop-in replacement for any convolutional layer. Experimental results for image classification on CIFAR-10 and ImageNet datasets using ResNet, MobileNetv2 and SeNet architectures substantiate the effectiveness of our proposed approach. We find that GKPD outperforms state-of-the-art decomposition methods including Tensor-Train and Tensor-Ring as well as other relevant compression methods such as pruning and knowledge distillation.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 29 Sep 2021.

Builds on

33 of 51 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2017Squeeze-and-Excitation Networkscited by 4,763
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2018SlowFast Networks for Video Recognitioncited by 1,862
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  9. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  10. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  11. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  12. 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
21 more
  1. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  2. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  3. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  4. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  5. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  6. 2013Predicting Parameters in Deep Learningcited by 223
  7. 2016Trained Ternary Quantizationcited by 203
  8. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  9. 2016Ternary Weight Networkscited by 195
  10. 2018Universality of Deep Convolutional Neural Networkscited by 176
  11. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  12. 2016Learning the Number of Neurons in Deep Networkscited by 69
  13. 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 55
  14. 2018Wide Compression: Tensor Ring Netscited by 50
  15. 2018Knowledge Distillation with Adversarial Samples Supporting Decision Boundarycited by 49
  16. 2017Compression-aware Training of Deep Networkscited by 46
  17. 2017Efficient Low Rank Tensor Ring Completioncited by 45
  18. 2019OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networkscited by 18
  19. 2019Compressing RNNs for IoT devices by 15-38x using Kronecker Productscited by 11
  20. 2015Exploiting Local Structures with the Kronecker Layer in Convolutional Networkscited by 7
  21. 2019FSNet: Compression of Deep Convolutional Neural Networks by Filter Summarycited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5; 2024: 2; 2025: 4.

14 papers.

Sort:

By area:

  1. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 130
  2. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  3. 2024DiffuseKronA: A Parameter Efficient Fine-tuning Method for Personalized Diffusion Modelscited by 11
  4. 2023Building Variable-sized Models via Learngene Poolcited by 7
  5. 2024Activation Map Compression through Tensor Decomposition for Deep Learningcited by 6
  6. 2023Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Trainingcited by 5
  7. 2025CAMEx: Curvature-aware Merging of Expertscited by 4
  8. 2023Understanding MLP-Mixer as a Wide and Sparse MLPcited by 3
  9. 2025An Efficient Training Algorithm for Models with Block-wise Sparsitycited by 2
  10. 2025Dynamic Rank Adjustment for Accurate and Efficient Neural Network Trainingcited by 1
  11. 2022Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architecturescited by 0
  12. 2022SeKron: A Decomposition Method Supporting Many Factorization Structurescited by 0
2 more
  1. 2023A priori compression of convolutional neural networks for wave simulatorscited by 0
  2. 2025Efficient Resource-Constrained Training of Transformers via Subspace Optimizationcited by 0