# Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework

arXiv: https://arxiv.org/abs/2206.08152 · HTML: https://mlsanity.com/paper/2206.08152/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 18 of 18)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Distributed Deep Neural Networks over the Cloud, the Edge and End Devices](https://mlsanity.com/paper/1709.01921/) (2017)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1](https://mlsanity.com/paper/1602.02830/) (2016)
- [JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1801.08618/) (2018)
- [JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution](https://mlsanity.com/paper/1812.10027/) (2018)
- [Supervised Compression for Resource-Constrained Edge Computing Systems](https://mlsanity.com/paper/2108.11898/) (2021)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device](https://mlsanity.com/paper/2104.09949/) (2021)
- [SplitNets: Designing Neural Architectures for Efficient Distributed Computing on Head-Mounted Systems](https://mlsanity.com/paper/2204.04705/) (2022)
- [Dynamic DNN Decomposition for Lossless Synergistic Inference](https://mlsanity.com/paper/2101.05952/) (2021)
- [Edge-PRUNE: Flexible Distributed Deep Learning Inference](https://mlsanity.com/paper/2204.12947/) (2022)

## Abstract (arXiv)

Collaborative inference has received significant research interest in machine learning as a vehicle for distributing computation load, reducing latency, as well as addressing privacy preservation in communications. Recent collaborative inference frameworks have adopted dynamic inference methodologies such as early-exit and run-time partitioning of neural networks. However, as machine learning frameworks scale in the number of inference inputs, e.g., in surveillance applications, fault tolerance related to device failure needs to be considered. This paper presents the Edge-PRUNE distributed computing framework, built on a formally defined model of computation, which provides a flexible infrastructure for fault tolerant collaborative inference. The experimental section of this work shows results on achievable inference time savings by collaborative inference, presents fault tolerant system topologies and analyzes their cost in terms of execution time overhead.