Exploiting Elasticity in Tensor Ranks for Compressing Neural Networks
Not on the map. Why?
Abstract (arXiv)
Elasticities in depth, width, kernel size and resolution have been explored in compressing deep neural networks (DNNs). Recognizing that the kernels in a convolutional neural network (CNN) are 4-way tensors, we further exploit a new elasticity dimension along the input-output channels. Specifically, a novel nuclear-norm rank minimization factorization (NRMF) approach is proposed to dynamically and globally search for the reduced tensor ranks during training. Correlation between tensor ranks across multiple layers is revealed, and a graceful tradeoff between model size and accuracy is obtained. Experiments then show the superiority of NRMF over the previous non-elastic variational Bayesian matrix factorization (VBMF) scheme.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 10 May 2021.
Builds on
17 of 30 references.
References · 17
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2014Going Deeper with Convolutionscited by 4,231
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
- 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
- 2015Tensorizing Neural Networkscited by 246
- 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
- 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 189
- 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
- 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
5 more
- 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 66
- 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 55
- 2019CBNet: A Novel Composite Backbone Network Architecture for Object Detectioncited by 44
- 2017CP-decomposition with Tensor Power Method for Convolutional Neural Networks Compressioncited by 30
- 2015Decoupled Deep Neural Network for Semi-supervised Semantic Segmentationcited by 23
Built on by
Citing papers per year: 2022: 1.
1 paper.