Approximating Continuous Convolutions for Deep Network Compression
Not on the map. Why?
Abstract (arXiv)
We present ApproxConv, a novel method for compressing the layers of a convolutional neural network. Reframing conventional discrete convolution as continuous convolution of parametrised functions over space, we use functional approximations to capture the essential structures of CNN filters with fewer parameters than conventional operations. Our method is able to reduce the size of trained CNN layers requiring only a small amount of fine-tuning. We show that our method is able to compress existing deep network models by half whilst losing only 1.86% accuracy. Further, we demonstrate that our method is compatible with other compression methods like quantisation allowing for further reductions in model size.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 17 Oct 2022.
Builds on
35 of 46 references.
References · 35
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2022A ConvNet for the 2020scited by 3,545
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
23 more
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2014FitNets: Hints for Thin Deep Netscited by 1,619
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
- 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
- 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
- 2017Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphscited by 307
- 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
- 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
- 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
- 2017SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernelscited by 160
- 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
- 2018Training DNNs with Hybrid Block Floating Pointcited by 39
- 2016Structured Receptive Fields in CNNscited by 32
- 2018Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)cited by 23
- 2017Computation Error Analysis of Block Floating Point Arithmetic Oriented Convolution Neural Network Accelerator Designcited by 7
- 2021Frequency learning for structured CNN filters with Gaussian fractional derivativescited by 6
- 2020Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processescited by 5
- 2019DSConv: Efficient Convolution Operatorcited by 4
Built on by
Not cited yet.