mlsanityData to 7 Oct 2026

Optimizing Convolutional Neural Network Architecture

arXiv 2401.01361 ↗First posted 17 Dec 2023cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Convolutional Neural Networks (CNN) are widely used to face challenging tasks like speech recognition, natural language processing or computer vision. As CNN architectures get larger and more complex, their computational requirements increase, incurring significant energetic costs and challenging their deployment on resource-restricted devices. In this paper, we propose Optimizing Convolutional Neural Network Architecture (OCNNA), a novel CNN optimization and construction method based on pruning and knowledge distillation designed to establish the importance of convolutional layers. The proposal has been evaluated though a thorough empirical study including the best known datasets (CIFAR-10, CIFAR-100 and Imagenet) and CNN architectures (VGG-16, ResNet-50, DenseNet-40 and MobileNet), setting Accuracy Drop and Remaining Parameters Ratio as objective metrics to compare the performance of OCNNA against the other state-of-art approaches. Our method has been compared with more than 20 convolutional neural network simplification algorithms obtaining outstanding results. As a result, OCNNA is a competitive CNN constructing method which could ease the deployment of neural networks into IoT or resource-limited devices.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 79 references that are in the map. First posted on arXiv 17 Dec 2023.

Builds on

34 of 79 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  11. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  12. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
22 more
  1. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  2. 2019Green AIcited by 620
  3. 2018Rethinking the Value of Network Pruningcited by 595
  4. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  5. 2017Large-Scale Evolution of Image Classifierscited by 348
  6. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  7. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  8. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  9. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  10. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  11. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  12. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  13. 2019Pruning from Scratchcited by 72
  14. 2020Discrimination-aware Network Pruning for Deep Model Compressioncited by 50
  15. 2021Carrying out CNN Channel Pruning in a White Boxcited by 22
  16. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  17. 2017Automatic Handgun Detection Alarm in Videos Using Deep Learningcited by 15
  18. 2020Breaking Batch Normalization for better explainability of Deep Neural Networks through Layer-wise Relevance Propagationcited by 11
  19. 2020Slimming Neural Networks using Adaptive Connectivity Scorescited by 6
  20. 2020Kernel Quantization for Efficient Network Compressioncited by 3
  21. 2020A Progressive Sub-Network Searching Framework for Dynamic Inferencecited by 2
  22. 2023Optimizing Dense Feed-Forward Neural Networkscited by 2

Built on by

Not cited yet.