mlsanityData to 14 Oct 2026

Lightweight Design and Optimization methods for DCNNs: Progress and Futures

arXiv 2412.16886 ↗First posted 22 Dec 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Lightweight design, as a key approach to mitigate disparity between computational requirements of deep learning models and hardware performance, plays a pivotal role in advancing application of deep learning technologies on mobile and embedded devices, alongside rapid development of smart home, telemedicine, and autonomous driving. With its outstanding feature extracting capabilities, Deep Convolutional Neural Networks (DCNNs) have demonstrated superior performance in computer vision tasks. However, high computational costs and large network architectures severely limit the widespread application of DCNNs on resource-constrained hardware platforms such as smartphones, robots, and IoT devices. This paper reviews lightweight design strategies for DCNNs and examines recent research progress in both lightweight architectural design and model compression. Additionally, this paper discusses current limitations in this field of research and propose prospects for future directions, aiming to provide valuable guidance and reflection for lightweight design philosophy on deep neural networks in the field of computer vision.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 62 references that are in the map. First posted on arXiv 22 Dec 2024.

Builds on

46 of 62 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  8. 2014Going Deeper with Convolutionscited by 4,241
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  12. 2019Searching for MobileNetV3cited by 1,994
34 more
  1. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  2. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  4. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  5. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  7. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
  8. 2018Object Detection with Deep Learning: A Reviewcited by 528
  9. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  10. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  11. 2019GhostNet: More Features from Cheap Operationscited by 410
  12. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 306
  13. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  14. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  15. 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
  16. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 203
  17. 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
  18. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  19. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 162
  20. 2017Sobolev Training for Neural Networkscited by 143
  21. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 143
  22. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 134
  23. 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
  24. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 89
  25. 2017Neural Optimizer Search with Reinforcement Learningcited by 82
  26. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  27. 2017DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfercited by 56
  28. 2017An Entropy-based Pruning Method for CNN Compressioncited by 37
  29. 2020AdderSR: Towards Energy Efficient Image Super-Resolutioncited by 32
  30. 2014Efficient and Accurate Approximations of Nonlinear Convolutional Networkscited by 30
  31. 2020Kernel Based Progressive Distillation for Adder Neural Networkscited by 19
  32. 2022RepGhost: A Hardware-Efficient Ghost Module via Re-parameterizationcited by 13
  33. 2018Extreme Network Compression via Filter Group Approximationcited by 12
  34. 2018A novel channel pruning method for deep neural network compressioncited by 8

Built on by

Not cited yet.