Computer Vision Model Compression Techniques for Embedded Systems: A Survey
Not on the map. Why?
Abstract (arXiv)
Deep neural networks have consistently represented the state of the art in most computer vision problems. In these scenarios, larger and more complex models have demonstrated superior performance to smaller architectures, especially when trained with plenty of representative data. With the recent adoption of Vision Transformer (ViT) based architectures and advanced Convolutional Neural Networks (CNNs), the total number of parameters of leading backbone architectures increased from 62M parameters in 2012 with AlexNet to 7B parameters in 2024 with AIM-7B. Consequently, deploying such deep architectures faces challenges in environments with processing and runtime constraints, particularly in embedded systems. This paper covers the main model compression techniques applied for computer vision tasks, enabling modern models to be used in embedded systems. We present the characteristics of compression subareas, compare different approaches, and discuss how to choose the best technique and expected variations when analyzing it on various embedded devices. We also share codes to assist researchers and new practitioners in overcoming initial implementation challenges for each subarea and present trends for Model Compression. Case studies for compression models are available at hrefhttps://github.com/venturusbr/cv-model-compressionhttps://github.com/venturusbr/cv-model-compression.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 92 of its 128 references that are in the map. First posted on arXiv 15 Aug 2024.
Builds on
92 of 128 references.
References · 92
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2014Going Deeper with Convolutionscited by 4,231
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2022A ConvNet for the 2020scited by 3,545
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2020Knowledge Distillation: A Surveycited by 1,578
38 more
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2016Pruning Filters for Efficient ConvNetscited by 1,129
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
- 2016Sequence-Level Knowledge Distillationcited by 761
- 2017Deep Mutual Learningcited by 621
- 2018Rethinking the Value of Network Pruningcited by 595
- 2020What is the State of Neural Network Pruning?cited by 578
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
- 2018Born Again Neural Networkscited by 539
- 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
- 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
- 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
- 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
- 2022Decoupled Knowledge Distillationcited by 380
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
- 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
- 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
- 2019Structured Knowledge Distillation for Dense Predictioncited by 277
- 2021Post-Training Quantization for Vision Transformercited by 274
- 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
- 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
- 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
- 2013Predicting Parameters in Deep Learningcited by 223
- 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
- 2022Knowledge Distillation from A Stronger Teachercited by 204
- 2023DepGraph: Towards Any Structural Pruningcited by 202
- 2023Structural Pruning for Diffusion Modelscited by 193
- 2022Language model compression with weighted low-rank factorizationcited by 192
- 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 188
- 2021Does Knowledge Distillation Really Work?cited by 187
- 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
- 2015Cross Modal Distillation for Supervision Transfercited by 173
- 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
- 2018Large scale distributed neural network training through online distillationcited by 144
50 most cited of 92 All 92 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.