mlsanityData to 7 Oct 2026

Federated Learning for Collaborative Inference Systems: The Case of Early Exit Networks

arXiv 2405.04249 ↗First posted 7 May 2024cs.LG cs.AI cs.DCcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

As Internet of Things (IoT) technology advances, end devices like sensors and smartphones are progressively equipped with AI models tailored to their local memory and computational constraints. Local inference reduces communication costs and latency; however, these smaller models typically underperform compared to more sophisticated models deployed on edge servers or in the cloud. Cooperative Inference Systems (CISs) address this performance trade-off by enabling smaller devices to offload part of their inference tasks to more capable devices. These systems often deploy hierarchical models that share numerous parameters, exemplified by Deep Neural Networks (DNNs) that utilize strategies like early exits or ordered dropout. In such instances, Federated Learning (FL) may be employed to jointly train the models within a CIS. Yet, traditional training methods have overlooked the operational dynamics of CISs during inference, particularly the potential high heterogeneity in serving rates across clients. To address this gap, we propose a novel FL approach designed explicitly for use in CISs that accounts for these variations in serving rates. Our framework not only offers rigorous theoretical guarantees, but also surpasses state-of-the-art (SOTA) training algorithms for CISs, especially in scenarios where inference request rates or data availability are uneven among clients.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 53 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

30 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges8 Mar

Other references · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  8. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  11. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  12. 2019Fair Resource Allocation in Federated Learningcited by 389
17 more
  1. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  2. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  3. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 150
  4. 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
  5. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 109
  6. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  7. 2019Improved Techniques for Training Adaptive Deep Networkscited by 66
  8. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 36
  9. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
  10. 2022Knowledge Distillation for Federated Learning: a Practical Guidecited by 31
  11. 2020Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networkscited by 29
  12. 2022A Survey on Collaborative DNN Inference for Edge Intelligencecited by 23
  13. 2023Federated Learning under Heterogeneous and Correlated Client Availabilitycited by 18
  14. 2023Federated Learning for Data Streamscited by 15
  15. 2016Optimal Quantile Approximation in Streamscited by 12
  16. 2022Over-the-Air Ensemble Inference with Model Privacycited by 9
  17. 2022Decentralized Low-Latency Collaborative Inference via Ensembles on the Edgecited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Hetero-SplitEE: Split Learning of Neural Networks with Early Exits for Heterogeneous IoT Devices
  2. 2025Collaborative Inference Acceleration with Non-Penetrative Tensor Partitioning
  3. 2023Adaptive Early Exiting for Collaborative Inference over Noisy Wireless Channels
  4. 2024Distributed Collaborative Inference System in Next-Generation Networks and Communication
  5. 2022FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers
  6. 2024Formal Logic Enabled Personalized Federated Learning Through Property Inference
  7. 2024Collaborative Inference for Large Models with Task Offloading and Early Exiting
  8. 2025DistrEE: Distributed Early Exit of Deep Neural Network Inference on Edge Devices