# Autodidactic Neurosurgeon: Collaborative Deep Inference for Mobile Edge Intelligence via Online Learning

arXiv: https://arxiv.org/abs/2102.02638 · HTML: https://mlsanity.com/paper/2102.02638/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1801.08618/) (2018)
- [BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1902.01000/) (2019)
- [Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy](https://mlsanity.com/paper/1806.07840/) (2018)
- [Deep Learning on Mobile Devices - A Review](https://mlsanity.com/paper/1904.09274/) (2019)
- [AdaLinUCB: Opportunistic Learning for Contextual Bandits](https://mlsanity.com/paper/1902.07802/) (2019)

## Built on by (most cited first, 8 of 8)

- [Holistic Network Virtualization and Pervasive Network Intelligence for 6G](https://mlsanity.com/paper/2301.00519/) (2023)
- [Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels](https://mlsanity.com/paper/2106.00995/) (2021)
- [AxiomVision: Accuracy-Guaranteed Adaptive Visual Model Selection for Perspective-Aware Video Analytics](https://mlsanity.com/paper/2407.20124/) (2024)
- [Learning the Optimal Path and DNN Partition for Collaborative Edge Inference](https://mlsanity.com/paper/2410.01857/) (2024)
- [Roulette: A Semantic Privacy-Preserving Device-Edge Collaborative Inference Framework for Deep Learning Classification Tasks](https://mlsanity.com/paper/2309.02820/) (2023)
- [t-READi: Transformer-Powered Robust and Efficient Multimodal Inference for Autonomous Driving](https://mlsanity.com/paper/2410.09747/) (2024)
- [When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Framework](https://mlsanity.com/paper/2305.12979/) (2023)
- [MANE: A Multi-Path Adaptive Network for Edge Onloading of Deep Neural Networks](https://mlsanity.com/paper/2609.14660/) (2026)

## Abstract (arXiv)

Recent breakthroughs in deep learning (DL) have led to the emergence of many intelligent mobile applications and services, but in the meanwhile also pose unprecedented computing challenges on resource-constrained mobile devices. This paper builds a collaborative deep inference system between a resource-constrained mobile device and a powerful edge server, aiming at joining the power of both on-device processing and computation offloading. The basic idea of this system is to partition a deep neural network (DNN) into a front-end part running on the mobile device and a back-end part running on the edge server, with the key challenge being how to locate the optimal partition point to minimize the end-to-end inference delay. Unlike existing efforts on DNN partitioning that rely heavily on a dedicated offline profiling stage to search for the optimal partition point, our system has a built-in online learning module, called Autodidactic Neurosurgeon (ANS), to automatically learn the optimal partition point on-the-fly. Therefore, ANS is able to closely follow the changes of the system environment by generating new knowledge for adaptive decision making. The core of ANS is a novel contextual bandit learning algorithm, called μLinUCB, which not only has provable theoretical learning performance guarantee but also is ultra-lightweight for easy real-world implementation. We implement our system on a video stream object detection testbed to validate the design of ANS and evaluate its performance. The experiments show that ANS significantly outperforms state-of-the-art benchmarks in terms of tracking system changes and reducing the end-to-end inference delay.