# Towards Collaborative Intelligence Friendly Architectures for Deep Learning

arXiv: https://arxiv.org/abs/1902.00147 · HTML: https://mlsanity.com/paper/1902.00147/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2019.

## Built on by (most cited first, 7 of 7)

- [Collaborative Intelligence: Challenges and Opportunities](https://mlsanity.com/paper/2102.06841/) (2021)
- [Lightweight compression of neural network feature tensors for collaborative intelligence](https://mlsanity.com/paper/2105.06002/) (2021)
- [Distributed CNN Inference on Resource-Constrained UAVs for Surveillance Systems: Design and Optimization](https://mlsanity.com/paper/2105.11013/) (2021)
- [Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence](https://mlsanity.com/paper/2105.07102/) (2021)
- [Single-Training Collaborative Object Detectors Adaptive to Bandwidth and Computation](https://mlsanity.com/paper/2105.00591/) (2021)
- [Latent-Space Inpainting for Packet Loss Concealment in Collaborative Object Detection](https://mlsanity.com/paper/2102.00142/) (2021)
- [Deep Edge Intelligence: Architecture, Key Features, Enabling Technologies and Challenges](https://mlsanity.com/paper/2210.12944/) (2022)

## Abstract (arXiv)

Modern mobile devices are equipped with high-performance hardware resources such as graphics processing units (GPUs), making the end-side intelligent services more feasible. Even recently, specialized silicons as neural engines are being used for mobile devices. However, most mobile devices are still not capable of performing real-time inference using very deep models. Computations associated with deep models for today's intelligent applications are typically performed solely on the cloud. This cloud-only approach requires significant amounts of raw data to be uploaded to the cloud over the mobile wireless network and imposes considerable computational and communication load on the cloud server. Recent studies have shown that the latency and energy consumption of deep neural networks in mobile applications can be notably reduced by splitting the workload between the mobile device and the cloud. In this approach, referred to as collaborative intelligence, intermediate features computed on the mobile device are offloaded to the cloud instead of the raw input data of the network, reducing the size of the data needed to be sent to the cloud. In this paper, we design a new collaborative intelligence friendly architecture by introducing a unit responsible for reducing the size of the feature data needed to be offloaded to the cloud to a greater extent, where this unit is placed after a selected layer of a deep model. Our proposed method, across different wireless networks, achieves on average 53x improvements for end-to-end latency and 68x improvements for mobile energy consumption compared to the status quo cloud-only approach for ResNet-50, while the accuracy loss is less than 2%.