# A Survey of Methods for Low-Power Deep Learning and Computer Vision

arXiv: https://arxiv.org/abs/2003.11066 · HTML: https://mlsanity.com/paper/2003.11066/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2020.

## Built on by (most cited first, 30 of 37)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [A Survey for Efficient Open Domain Question Answering](https://mlsanity.com/paper/2211.07886/) (2022)
- [Fixing Overconfidence in Dynamic Neural Networks](https://mlsanity.com/paper/2302.06359/) (2023)
- [Challenges and Practices of Deep Learning Model Reengineering: A Case Study on Computer Vision](https://mlsanity.com/paper/2303.07476/) (2023)
- [Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework](https://mlsanity.com/paper/2411.18376/) (2024)
- [Guaranteed Approximation Bounds for Mixed-Precision Neural Operators](https://mlsanity.com/paper/2307.15034/) (2023)
- [Pruning One More Token is Enough: Leveraging Latency-Workload Non-Linearities for Vision Transformers on the Edge](https://mlsanity.com/paper/2407.05941/) (2024)
- [Spiking-GAN: A Spiking Generative Adversarial Network Using Time-To-First-Spike Coding](https://mlsanity.com/paper/2106.15420/) (2021)
- [Efficient Adaptive Ensembling for Image Classification](https://mlsanity.com/paper/2206.07394/) (2022)
- [Flexible and Hierarchical Prior for Bayesian Nonnegative Matrix Factorization](https://mlsanity.com/paper/2205.11025/) (2022)
- [Spiking-Diffusion: Vector Quantized Discrete Diffusion Model with Spiking Neural Networks](https://mlsanity.com/paper/2308.10187/) (2023)
- [DSD2: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?](https://mlsanity.com/paper/2303.01213/) (2023)
- [Bayesian Low-Rank Interpolative Decomposition for Complex Datasets](https://mlsanity.com/paper/2205.14825/) (2022)
- [A Plug-in Tiny AI Module for Intelligent and Selective Sensor Data Transmission](https://mlsanity.com/paper/2402.02043/) (2024)
- [Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving](https://mlsanity.com/paper/2110.03553/) (2021)
- [BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration](https://mlsanity.com/paper/2411.01417/) (2024)
- [Efficient Computer Vision on Edge Devices with Pipeline-Parallel Hierarchical Neural Networks](https://mlsanity.com/paper/2109.13356/) (2021)
- [Software Engineering Approaches for TinyML based IoT Embedded Vision: A Systematic Literature Review](https://mlsanity.com/paper/2204.08702/) (2022)
- [Comparative Study of Inference Methods for Interpolative Decomposition](https://mlsanity.com/paper/2206.14542/) (2022)
- [Impact of ML Optimization Tactics on Greener Pre-Trained ML Models](https://mlsanity.com/paper/2409.12878/) (2024)
- [Reducing Inference Energy Consumption Using Dual Complementary CNNs](https://mlsanity.com/paper/2412.01039/) (2024)
- [Low-Power Multi-Camera Object Re-Identification using Hierarchical Neural Networks](https://mlsanity.com/paper/2106.10588/) (2021)
- [Chemical transformer compression for accelerating both training and inference of molecular modeling](https://mlsanity.com/paper/2205.07582/) (2022)
- [Computer Vision Model Compression Techniques for Embedded Systems: A Survey](https://mlsanity.com/paper/2408.08250/) (2024)
- [Towards End-to-End Neural Face Authentication in the Wild -- Quantifying and Compensating for Directional Lighting Effects](https://mlsanity.com/paper/2104.03854/) (2021)
- [Efficient visual object representation using a biologically plausible spike-latency code and winner-take-all inhibition](https://mlsanity.com/paper/2205.10338/) (2022)
- [Land Classification in Satellite Images by Injecting Traditional Features to CNN Models](https://mlsanity.com/paper/2207.10368/) (2022)
- [Application for White Spot Syndrome Virus (WSSV) Monitoring using Edge Machine Learning](https://mlsanity.com/paper/2308.04151/) (2023)
- [On the fly Deep Neural Network Optimization Control for Low-Power Computer Vision](https://mlsanity.com/paper/2309.01824/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) are successful in many computer vision tasks. However, the most accurate DNNs require millions of parameters and operations, making them energy, computation and memory intensive. This impedes the deployment of large DNNs in low-power devices with limited compute resources. Recent research improves DNN models by reducing the memory requirement, energy consumption, and number of operations without significantly decreasing the accuracy. This paper surveys the progress of low-power deep learning and computer vision, specifically in regards to inference, and discusses the methods for compacting and accelerating DNN models. The techniques can be divided into four major categories: (1) parameter quantization and pruning, (2) compressed convolutional filters and matrix factorization, (3) network architecture search, and (4) knowledge distillation. We analyze the accuracy, advantages, disadvantages, and potential solutions to the problems with the techniques in each category. We also discuss new evaluation metrics as a guideline for future research.