# Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

arXiv: https://arxiv.org/abs/1910.05316 · HTML: https://mlsanity.com/paper/1910.05316/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2019.

## Built on by (most cited first, 30 of 152)

- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [Dynamic Sparse Training with Structured Sparsity](https://mlsanity.com/paper/2305.02299/) (2023)
- [Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directions](https://mlsanity.com/paper/2208.03392/) (2022)
- [Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI](https://mlsanity.com/paper/2207.00969/) (2022)
- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Vision-Language Models for Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/2502.07855/) (2025)
- [Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization](https://mlsanity.com/paper/2110.10030/) (2021)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference](https://mlsanity.com/paper/2206.04685/) (2022)
- [A Survey on Collaborative DNN Inference for Edge Intelligence](https://mlsanity.com/paper/2207.07812/) (2022)
- [DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device](https://mlsanity.com/paper/2104.09949/) (2021)
- [Accelerating Split Federated Learning over Wireless Communication Networks](https://mlsanity.com/paper/2310.15584/) (2023)
- [Task-Oriented Over-the-Air Computation for Multi-Device Edge AI](https://mlsanity.com/paper/2211.01255/) (2022)
- [Holistic Network Virtualization and Pervasive Network Intelligence for 6G](https://mlsanity.com/paper/2301.00519/) (2023)
- [SplitEE: Early Exit in Deep Neural Networks with Split Computing](https://mlsanity.com/paper/2309.09195/) (2023)
- [Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router](https://mlsanity.com/paper/2506.05901/) (2025)
- [CEEBERT: Cross-Domain Inference in Early Exit BERT](https://mlsanity.com/paper/2405.15039/) (2024)
- [Adaptive Layer Splitting for Wireless LLM Inference in Edge Computing: A Model-Based Reinforcement Learning Approach](https://mlsanity.com/paper/2406.02616/) (2024)
- [SplitPlace: AI Augmented Splitting and Placement of Large-Scale Neural Networks in Mobile Edge Environments](https://mlsanity.com/paper/2205.10635/) (2022)
- [Multi-user Co-inference with Batch Processing Capable Edge Server](https://mlsanity.com/paper/2206.06304/) (2022)
- [Resource-Constrained Edge AI with Early Exit Prediction](https://mlsanity.com/paper/2206.07269/) (2022)
- [Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning](https://mlsanity.com/paper/2209.02428/) (2022)
- [GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Servers](https://mlsanity.com/paper/2210.17281/) (2022)
- [Unsupervised Early Exit in DNNs with Multiple Exits](https://mlsanity.com/paper/2209.09480/) (2022)
- [Accuracy-Guaranteed Collaborative DNN Inference in Industrial IoT via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.00130/) (2022)
- [Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices](https://mlsanity.com/paper/2102.06336/) (2021)
- [Autodidactic Neurosurgeon: Collaborative Deep Inference for Mobile Edge Intelligence via Online Learning](https://mlsanity.com/paper/2102.02638/) (2021)
- [Complexity-Driven CNN Compression for Resource-constrained Edge AI](https://mlsanity.com/paper/2208.12816/) (2022)

## Abstract (arXiv)

As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment, Edgentis properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.