mlsanityData to 7 Oct 2026

SeKron: A Decomposition Method Supporting Many Factorization Structures

arXiv 2210.06299 ↗First posted 12 Oct 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

While convolutional neural networks (CNNs) have become the de facto standard for most image processing and computer vision applications, their deployment on edge devices remains challenging. Tensor decomposition methods provide a means of compressing CNNs to meet the wide range of device constraints by imposing certain factorization structures on their convolution tensors. However, being limited to the small set of factorization structures presented by state-of-the-art decomposition approaches can lead to sub-optimal performance. We propose SeKron, a novel tensor decomposition method that offers a wide variety of factorization structures, using sequences of Kronecker products. By recursively finding approximating Kronecker factors, we arrive at optimal decompositions for each of the factorization structures. We show that SeKron is a flexible decomposition that generalizes widely used methods, such as Tensor-Train (TT), Tensor-Ring (TR), Canonical Polyadic (CP) and Tucker decompositions. Crucially, we derive an efficient convolution projection algorithm shared by all SeKron structures, leading to seamless compression of CNN models. We validate SeKron for model compression on both high-level and low-level computer vision tasks and find that it outperforms state-of-the-art decomposition methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 56 of its 77 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

56 of 77 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2016Wide Residual Networkscited by 3,057
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  10. 2019Searching for MobileNetV3cited by 1,984
  11. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  12. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
38 more
  1. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
  2. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  3. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  4. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  5. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  7. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  8. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  9. 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
  10. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  11. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  12. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  13. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  14. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  15. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  16. 2015Tensorizing Neural Networkscited by 246
  17. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  18. 2013Predicting Parameters in Deep Learningcited by 223
  19. 2016Trained Ternary Quantizationcited by 203
  20. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  21. 2016Ternary Weight Networkscited by 195
  22. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  23. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  24. 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 118
  25. 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
  26. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  27. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  28. 2020Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Networkcited by 74
  29. 2014Learning to Deblurcited by 66
  30. 2019Learning Filter Basis for Convolutional Neural Network Compressioncited by 57
  31. 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 55
  32. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  33. 2018Wide Compression: Tensor Ring Netscited by 50
  34. 2018Knowledge Distillation with Adversarial Samples Supporting Decision Boundarycited by 49
  35. 2017Compression-aware Training of Deep Networkscited by 46
  36. 2017Efficient Low Rank Tensor Ring Completioncited by 45
  37. 2018Generalized Canonical Polyadic Tensor Decompositioncited by 34
  38. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.