# Auto-Split: A General Framework of Collaborative Edge-Cloud AI

arXiv: https://arxiv.org/abs/2108.13041 · HTML: https://mlsanity.com/paper/2108.13041/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 59 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications](https://mlsanity.com/paper/1804.03230/) (2018)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy](https://mlsanity.com/paper/1711.05852/) (2017)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation](https://mlsanity.com/paper/1812.08934/) (2018)
- [Post-training 4-bit quantization of convolution networks for rapid-deployment](https://mlsanity.com/paper/1810.05723/) (2018)
- [PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors](https://mlsanity.com/paper/1908.11263/) (2019)

## Built on by (most cited first, 30 of 30)

- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [SC2 Benchmark: Supervised Compression for Split Computing](https://mlsanity.com/paper/2203.08875/) (2022)
- [Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey](https://mlsanity.com/paper/2505.01821/) (2025)
- [Bounding the Invertibility of Privacy-preserving Instance Encoding using Fisher Information](https://mlsanity.com/paper/2305.04146/) (2023)
- [Device-Cloud Collaborative Recommendation via Meta Controller](https://mlsanity.com/paper/2207.03066/) (2022)
- [E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models](https://mlsanity.com/paper/2203.00748/) (2022)
- [Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization](https://mlsanity.com/paper/2501.12388/) (2024)
- [Privacy-Preserving Visual Localization with Event Cameras](https://mlsanity.com/paper/2212.03177/) (2022)
- [TinyMLOps: Operational Challenges for Widespread Edge AI Adoption](https://mlsanity.com/paper/2203.10923/) (2022)
- [Measuring and Controlling Split Layer Privacy Leakage Using Fisher Information](https://mlsanity.com/paper/2209.10119/) (2022)
- [Salted Inference: Enhancing Privacy while Maintaining Efficiency of Split Inference in Mobile Computing](https://mlsanity.com/paper/2310.13384/) (2023)
- [DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation](https://mlsanity.com/paper/2406.08804/) (2024)
- [A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devices](https://mlsanity.com/paper/2407.03331/) (2024)
- [A Survey on Cloud-Edge-Terminal Collaborative Intelligence in AIoT Networks](https://mlsanity.com/paper/2508.18803/) (2025)
- [Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems](https://mlsanity.com/paper/2404.13348/) (2024)
- [Efficient Distributed Retrieval-Augmented Generation for Enhancing Language Model Performance](https://mlsanity.com/paper/2504.11197/) (2025)
- [Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearables](https://mlsanity.com/paper/2401.08637/) (2023)
- [On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2411.00907/) (2024)
- [Distributed Collaborative Inference System in Next-Generation Networks and Communication](https://mlsanity.com/paper/2412.12102/) (2024)
- [MEL: Multi-level Ensemble Learning for Resource-Constrained Environments](https://mlsanity.com/paper/2506.20094/) (2025)
- [Intra-DP: A High Performance Collaborative Inference System for Mobile Edge Computing](https://mlsanity.com/paper/2507.05829/) (2025)
- [Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework](https://mlsanity.com/paper/2206.08152/) (2022)
- [Towards A Flexible Accuracy-Oriented Deep Learning Module Inference Latency Prediction Framework for Adaptive Optimization Algorithms](https://mlsanity.com/paper/2312.06440/) (2023)
- [Edge AI as a Service with Coordinated Deep Neural Networks](https://mlsanity.com/paper/2401.00631/) (2024)
- [Optimal In-Network Distribution of Learning Functions for a Secure-by-Design Programmable Data Plane of Next-Generation Networks](https://mlsanity.com/paper/2411.18384/) (2024)
- [Revisiting the Privacy Risks of Split Inference: A GAN-Based Data Reconstruction Attack via Progressive Feature Optimization](https://mlsanity.com/paper/2508.20613/) (2025)
- [CoLM: Collaborative Large Models via A Client-Server Paradigm](https://mlsanity.com/paper/2511.06991/) (2025)
- [Trainable Bitwise Soft Quantization for Input Feature Compression](https://mlsanity.com/paper/2603.05172/) (2026)
- [AdaDINO: Context-Adaptive DINO-Distilled Vision Foundation Models for Efficient Open-Vocabulary Edge Inference](https://mlsanity.com/paper/2604.15622/) (2026)
- [Parameter- and Bandwidth-Efficient Edge--cloud Many-to-Many Speech-to-Text Translation](https://mlsanity.com/paper/2605.28642/) (2026)

## Abstract (arXiv)

In many industry scale applications, large and resource consuming machine learning models reside in powerful cloud servers. At the same time, large amounts of input data are collected at the edge of cloud. The inference results are also communicated to users or passed to downstream tasks at the edge. The edge often consists of a large number of low-power devices. It is a big challenge to design industry products to support sophisticated deep model deployment and conduct model inference in an efficient manner so that the model accuracy remains high and the end-to-end latency is kept low. This paper describes the techniques and engineering practice behind Auto-Split, an edge-cloud collaborative prototype of Huawei Cloud. This patented technology is already validated on selected applications, is on its way for broader systematic edge-cloud application integration, and is being made available for public use as an automated pipeline service for end-to-end cloud-edge collaborative intelligence deployment. To the best of our knowledge, there is no existing industry product that provides the capability of Deep Neural Network (DNN) splitting.