Learning Versatile Convolution Filters for Efficient Visual Recognition
Not on the map. Why?
Abstract (arXiv)
This paper introduces versatile filters to construct efficient convolutional neural networks that are widely used in various visual recognition tasks. Considering the demands of efficient deep learning techniques running on cost-effective hardware, a number of methods have been developed to learn compact neural networks. Most of these works aim to slim down filters in different ways, eg,~investigating small, sparse or quantized filters. In contrast, we treat filters from an additive perspective. A series of secondary filters can be derived from a primary filter with the help of binary masks. These secondary filters all inherit in the primary filter without occupying more storage, but once been unfolded in computation they could significantly enhance the capability of the filter by integrating information extracted from different receptive fields. Besides spatial versatile filters, we additionally investigate versatile filters from the channel perspective. Binary masks can be further customized for different primary filters under orthogonal constraints. We conduct theoretical analysis on network complexity and an efficient convolution scheme is introduced. Experimental results on benchmark datasets and neural networks demonstrate that our versatile filters are able to achieve comparable accuracy as that of original filters, but require less memory and computation cost.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 69 of its 85 references that are in the map. First posted on arXiv 20 Sep 2021.
Builds on
69 of 85 references.
References · 69
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2014Going Deeper with Convolutionscited by 4,231
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
38 more
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2019Searching for MobileNetV3cited by 1,984
- 2014FitNets: Hints for Thin Deep Netscited by 1,619
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2016Pruning Filters for Efficient ConvNetscited by 1,129
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
- 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
- 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
- 2019Importance Estimation for Neural Network Pruningcited by 531
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
- 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
- 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
- 2019GhostNet: More Features from Cheap Operationscited by 410
- 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
- 2018Slimmable Neural Networkscited by 301
- 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
- 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 287
- 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
- 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
- 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
- 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
- 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
- 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 212
- 2019Data-Free Learning of Student Networkscited by 203
- 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
50 most cited of 69 All 69 (JSON)
Built on by
Citing papers per year: 2022: 2; 2023: 1.
3 papers.