mlsanityData to 7 Oct 2026

SplitEE: Early Exit in Deep Neural Networks with Split Computing

arXiv 2309.09195 ↗First posted 17 Sep 2023cs.LG cs.AI cs.CLcited by 13

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Deep Neural Networks (DNNs) have drawn attention because of their outstanding performance on various tasks. However, deploying full-fledged DNNs in resource-constrained devices (edge, mobile, IoT) is difficult due to their large size. To overcome the issue, various approaches are considered, like offloading part of the computation to the cloud for final inference (split computing) or performing the inference at an intermediary layer without passing through all layers (early exits). In this work, we propose combining both approaches by using early exits in split computing. In our approach, we decide up to what depth of DNNs computation to perform on the device (splitting layer) and whether a sample can exit from this layer or need to be offloaded. The decisions are based on a weighted combination of accuracy, computational, and communication costs. We develop an algorithm named SplitEE to learn an optimal policy. Since pre-trained DNNs are often deployed in new domains where the ground truths may be unavailable and samples arrive in a streaming fashion, SplitEE works in an online and unsupervised setup. We extensively perform experiments on five different datasets. SplitEE achieves a significant cost reduction (>50%) with a slight drop in accuracy (<2%) as compared to the case when all samples are inferred at the final layer. The anonymized source code is available at urlhttps://anonymous.4open.science/r/SplitEEM-B989/README.md.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 19 of its 41 references that are in the map. First posted on arXiv 17 Sep 2023.

Builds on

19 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Early-exit deep neural networks for distorted images: providing an efficient edge offloading20 Aug
  2. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges8 Mar

Other references · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2018Deep contextualized word representationscited by 2,285
  6. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  7. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  8. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  9. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 150
  10. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 109
  11. 2019BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscited by 94
  12. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
5 more
  1. 2019BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Servicescited by 67
  2. 2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
  3. 2020Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networkscited by 28
  4. 2022BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computingcited by 28
  5. 2022Unsupervised Early Exit in DNNs with Multiple Exitscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 2.

13 papers.

Sort:

By area:

  1. 2024CEEBERT: Cross-Domain Inference in Early Exit BERTcited by 12
  2. 2025FREE: Fast and Robust Vision Language Models with Early Exitscited by 12
  3. 2025BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Expertscited by 6
  4. 2024Distributed Inference on Mobile Edge and Cloud: An Early Exit based Clustering Approachcited by 3
  5. 2025Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliabilitycited by 3
  6. 2025Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computingcited by 3
  7. 2025FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inferencecited by 1
  8. 2024Distributed Inference on Mobile Edge and Cloud: A Data-Cartography based Clustering Approachcited by 0
  9. 2025Know What You Don't Know: Selective Prediction for Early Exit DNNscited by 0
  10. 2025Range Asymmetric Numeral Systems-Based Lightweight Intermediate Feature Compression for Split Computing of Deep Neural Networkscited by 0
  11. 2025Hetero-SplitEE: Split Learning of Neural Networks with Early Exits for Heterogeneous IoT Devicescited by 0
  12. 2026A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networkscited by 0
1 more
  1. 2026SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024I-SplitEE: Image classification in Split Computing DNNs with Early Exits
  2. 2024Enhancing Split Computing and Early Exit Applications through Predefined Sparsity
  3. 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
  4. 2022Resource-Constrained Edge AI with Early Exit Prediction
  5. 2022Pruning Early Exit Networks
  6. 2025DistrEE: Distributed Early Exit of Deep Neural Network Inference on Edge Devices
  7. 2024Early-Exit meets Model-Distributed Inference at Edge Networks
  8. 2023Hierarchical Training of Deep Neural Networks Using Early Exiting