# SplitEE: Early Exit in Deep Neural Networks with Split Computing arXiv: https://arxiv.org/abs/2309.09195 ยท HTML: https://mlsanity.com/paper/2309.09195/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. - Builds on 19 of its 41 references that are in the map. - First posted on arXiv 17 Sep 2023. ## Placed in - Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/) ## Builds on (most cited first, 19 of 19) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018) - [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019) - [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018) - [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017) - [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020) - [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020) - [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019) - [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021) - [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020) - [BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems](https://mlsanity.com/paper/1910.14315/) (2019) - [Yelp Dataset Challenge: Review Rating Prediction](https://mlsanity.com/paper/1605.05362/) (2016) - [BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1902.01000/) (2019) - [Towards Efficient NLP: A Standard Evaluation and A Strong Baseline](https://mlsanity.com/paper/2110.07038/) (2021) - [Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks](https://mlsanity.com/paper/2007.15818/) (2020) - [BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing](https://mlsanity.com/paper/2201.02693/) (2022) - [Unsupervised Early Exit in DNNs with Multiple Exits](https://mlsanity.com/paper/2209.09480/) (2022) - [Early-exit deep neural networks for distorted images: providing an efficient edge offloading](https://mlsanity.com/paper/2108.09343/) (2021) ## Built on by (most cited first, 13 of 13) - [CEEBERT: Cross-Domain Inference in Early Exit BERT](https://mlsanity.com/paper/2405.15039/) (2024) - [FREE: Fast and Robust Vision Language Models with Early Exits](https://mlsanity.com/paper/2506.06884/) (2025) - [BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts](https://mlsanity.com/paper/2502.00745/) (2025) - [Distributed Inference on Mobile Edge and Cloud: An Early Exit based Clustering Approach](https://mlsanity.com/paper/2410.05338/) (2024) - [Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability](https://mlsanity.com/paper/2509.23666/) (2025) - [Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing](https://mlsanity.com/paper/2511.04002/) (2025) - [FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference](https://mlsanity.com/paper/2510.22641/) (2025) - [Distributed Inference on Mobile Edge and Cloud: A Data-Cartography based Clustering Approach](https://mlsanity.com/paper/2412.16616/) (2024) - [Know What You Don't Know: Selective Prediction for Early Exit DNNs](https://mlsanity.com/paper/2509.11520/) (2025) - [Range Asymmetric Numeral Systems-Based Lightweight Intermediate Feature Compression for Split Computing of Deep Neural Networks](https://mlsanity.com/paper/2511.11664/) (2025) - [Hetero-SplitEE: Split Learning of Neural Networks with Early Exits for Heterogeneous IoT Devices](https://mlsanity.com/paper/2512.09313/) (2025) - [A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networks](https://mlsanity.com/paper/2604.24810/) (2026) - [SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference](https://mlsanity.com/paper/2608.13076/) (2026) ## Abstract (arXiv) Deep Neural Networks (DNNs) have drawn attention because of their outstanding performance on various tasks. However, deploying full-fledged DNNs in resource-constrained devices (edge, mobile, IoT) is difficult due to their large size. To overcome the issue, various approaches are considered, like offloading part of the computation to the cloud for final inference (split computing) or performing the inference at an intermediary layer without passing through all layers (early exits). In this work, we propose combining both approaches by using early exits in split computing. In our approach, we decide up to what depth of DNNs computation to perform on the device (splitting layer) and whether a sample can exit from this layer or need to be offloaded. The decisions are based on a weighted combination of accuracy, computational, and communication costs. We develop an algorithm named SplitEE to learn an optimal policy. Since pre-trained DNNs are often deployed in new domains where the ground truths may be unavailable and samples arrive in a streaming fashion, SplitEE works in an online and unsupervised setup. We extensively perform experiments on five different datasets. SplitEE achieves a significant cost reduction (>50%) with a slight drop in accuracy (<2%) as compared to the case when all samples are inferred at the final layer. The anonymized source code is available at urlhttps://anonymous.4open.science/r/SplitEEM-B989/README.md.