# Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

arXiv: https://arxiv.org/abs/1806.07840 · HTML: https://mlsanity.com/paper/1806.07840/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2018.

## Built on by (most cited first, 30 of 51)

- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Auditing Membership Leakages of Multi-Exit Networks](https://mlsanity.com/paper/2208.11180/) (2022)
- [FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning](https://mlsanity.com/paper/2107.04271/) (2021)
- [At the Dawn of Generative AI Era: A Tutorial-cum-Survey on New Frontiers in 6G Wireless Intelligence](https://mlsanity.com/paper/2402.18587/) (2024)
- [A Survey on Collaborative DNN Inference for Edge Intelligence](https://mlsanity.com/paper/2207.07812/) (2022)
- [LegoDNN: Block-grained Scaling of Deep Neural Networks for Mobile Vision](https://mlsanity.com/paper/2112.09852/) (2021)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [Learning to Cascade: Confidence Calibration for Improving the Accuracy and Computational Cost of Cascade Inference Systems](https://mlsanity.com/paper/2104.09286/) (2021)
- [Autodidactic Neurosurgeon: Collaborative Deep Inference for Mobile Edge Intelligence via Online Learning](https://mlsanity.com/paper/2102.02638/) (2021)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Edge-Cloud Collaborated Object Detection via Difficult-Case Discriminator](https://mlsanity.com/paper/2108.12858/) (2021)
- [Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization](https://mlsanity.com/paper/2501.12388/) (2024)
- [NetCut: Real-Time DNN Inference Using Layer Removal](https://mlsanity.com/paper/2101.05363/) (2021)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [DynaMarks: Defending Against Deep Learning Model Extraction Using Dynamic Watermarking](https://mlsanity.com/paper/2207.13321/) (2022)
- [Knowledge boosting during low-latency inference](https://mlsanity.com/paper/2407.11055/) (2024)
- [JMSNAS: Joint Model Split and Neural Architecture Search for Learning over Mobile Edge Networks](https://mlsanity.com/paper/2111.08206/) (2021)
- [A Survey on Deep Neural Network Partition over Cloud, Edge and End Devices](https://mlsanity.com/paper/2304.10020/) (2023)
- [On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2403.17154/) (2024)
- [BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration](https://mlsanity.com/paper/2411.01417/) (2024)
- [The Cognitive Firewall:Securing Browser Based AI Agents Against Indirect Prompt Injection Via Hybrid Edge Cloud Defense](https://mlsanity.com/paper/2603.23791/) (2026)
- [Latency-Memory Optimized Splitting of Convolution Neural Networks for Resource Constrained Edge Devices](https://mlsanity.com/paper/2107.09123/) (2021)
- [Iterative Activation-based Structured Pruning](https://mlsanity.com/paper/2201.09881/) (2022)
- [Roulette: A Semantic Privacy-Preserving Device-Edge Collaborative Inference Framework for Deep Learning Classification Tasks](https://mlsanity.com/paper/2309.02820/) (2023)
- [AutoDiCE: Fully Automated Distributed CNN Inference at the Edge](https://mlsanity.com/paper/2207.12113/) (2022)
- [Edge AI for Internet of Energy: Challenges and Perspectives](https://mlsanity.com/paper/2311.16851/) (2023)
- [A Converting Autoencoder Toward Low-latency and Energy-efficient DNN Inference at the Edge](https://mlsanity.com/paper/2403.07036/) (2024)
- [On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2411.00907/) (2024)
- [Aligning Task- and Reconstruction-Oriented Communications for Edge Intelligence](https://mlsanity.com/paper/2502.15472/) (2025)
- [DistrEE: Distributed Early Exit of Deep Neural Network Inference on Edge Devices](https://mlsanity.com/paper/2502.15735/) (2025)

## Abstract (arXiv)

As the backbone technology of machine learning, deep neural networks (DNNs) have have quickly ascended to the spotlight. Running DNNs on resource-constrained mobile devices is, however, by no means trivial, since it incurs high performance and energy overhead. While offloading DNNs to the cloud for execution suffers unpredictable performance, due to the uncontrolled long wide-area network latency. To address these challenges, in this paper, we propose Edgent, a collaborative and on-demand DNN co-inference framework with device-edge synergy. Edgent pursues two design knobs: (1) DNN partitioning that adaptively partitions DNN computation between device and edge, in order to leverage hybrid computation resources in proximity for real-time DNN inference. (2) DNN right-sizing that accelerates DNN inference through early-exit at a proper intermediate DNN layer to further reduce the computation latency. The prototype implementation and extensive evaluations based on Raspberry Pi demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.