# Efficient Computer Vision on Edge Devices with Pipeline-Parallel Hierarchical Neural Networks

arXiv: https://arxiv.org/abs/2109.13356 · HTML: https://mlsanity.com/paper/2109.13356/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 27 Sep 2021.

## Builds on (most cited first, 10 of 10)

- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective](https://mlsanity.com/paper/1811.03402/) (2018)
- [A Survey of Methods for Low-Power Deep Learning and Computer Vision](https://mlsanity.com/paper/2003.11066/) (2020)
- [Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning](https://mlsanity.com/paper/1802.05800/) (2018)
- [Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT](https://mlsanity.com/paper/1907.11804/) (2019)
- [Hardware-Aware Machine Learning: Modeling and Optimization](https://mlsanity.com/paper/1809.05476/) (2018)
- [Low-Power Object Counting with Hierarchical Neural Networks](https://mlsanity.com/paper/2007.01369/) (2020)
- [Collaborative Intelligent Cross-Camera Video Analytics at Edge: Opportunities and Challenges](https://mlsanity.com/paper/1909.10468/) (2019)
- [FALCON: Feature Driven Selective Classification for Energy-Efficient Image Recognition](https://mlsanity.com/paper/1609.03396/) (2016)
- [Low-Power Multi-Camera Object Re-Identification using Hierarchical Neural Networks](https://mlsanity.com/paper/2106.10588/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Survey on Collaborative DNN Inference for Edge Intelligence](https://mlsanity.com/paper/2207.07812/) (2022)
- [Adaptive Workload Distribution for Accuracy-aware DNN Inference on Collaborative Edge Platforms](https://mlsanity.com/paper/2310.10157/) (2023)

## Abstract (arXiv)

Computer vision on low-power edge devices enables applications including search-and-rescue and security. State-of-the-art computer vision algorithms, such as Deep Neural Networks (DNNs), are too large for inference on low-power edge devices. To improve efficiency, some existing approaches parallelize DNN inference across multiple edge devices. However, these techniques introduce significant communication and synchronization overheads or are unable to balance workloads across devices. This paper demonstrates that the hierarchical DNN architecture is well suited for parallel processing on multiple edge devices. We design a novel method that creates a parallel inference pipeline for computer vision problems that use hierarchical DNNs. The method balances loads across the collaborating devices and reduces communication costs to facilitate the processing of multiple video frames simultaneously with higher throughput. Our experiments consider a representative computer vision problem where image recognition is performed on each video frame, running on multiple Raspberry Pi 4Bs. With four collaborating low-power edge devices, our approach achieves 3.21X higher throughput, 68% less energy consumption per device per frame, and 58% decrease in memory when compared with existing single-device hierarchical DNNs.