mlsanityData to 7 Oct 2026

Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization

arXiv 2203.09747 ↗First posted 18 Mar 2022cs.LG cs.DCcited by 34

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In many practical FL scenarios, participants have heterogeneous resources due to disparities in hardware and inference dynamics that require quickly loading models of different sizes and levels of robustness. The heterogeneity and dynamics together impose significant challenges to existing FL approaches and thus greatly limit FL's applicability. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness. Specifically, we achieve customization by learning a set of base sub-networks of different sizes and robustness levels, which are later aggregated on-demand according to inference requirements. This split-mix strategy achieves customization with high efficiency in communication, storage, and inference. Extensive experiments demonstrate that our method provides better in-situ customization than the existing heterogeneous-architecture FL methods. Codes and pre-trained models are available: https://github.com/illidanlab/SplitMix.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

30 of 49 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2016Wide Residual Networkscited by 3,057
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  11. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
  12. 2018Robustness May Be at Odds with Accuracycited by 873
18 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 780
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  3. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  4. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  5. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  6. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  7. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  8. 2018Slimmable Neural Networkscited by 301
  9. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  10. 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
  11. 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 185
  12. 2019Federated Adversarial Domain Adaptationcited by 140
  13. 2021AugMax: Adversarial Composition of Random Augmentations for Robust Trainingcited by 89
  14. 2018AI Benchmark: Running Deep Neural Networks on Android Smartphonescited by 75
  15. 2017Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Executioncited by 59
  16. 2020Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Freecited by 48
  17. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  18. 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 12; 2024: 8; 2025: 3; 2026: 3.

34 papers.

Sort:

By area:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  2. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  3. 2022A Survey on Heterogeneous Federated Learningcited by 42
  4. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  5. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  6. 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 23
  7. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  8. 2022Federated Learning of Large Models at the Edge via Principal Sub-Model Trainingcited by 13
  9. 2022GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Serverscited by 11
  10. 2024Recurrent Early Exits for Federated Learning with Heterogeneous Clientscited by 11
  11. 2023NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clientscited by 7
  12. 2023Personalised Federated Learning On Heterogeneous Feature Spacescited by 5
22 more
  1. 2023AdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systemscited by 4
  2. 2024FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learningcited by 4
  3. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  4. 2022Outsourcing Training without Uploading Data via Efficient Collaborative Open-Source Samplingcited by 3
  5. 2022HierarchyFL: Heterogeneous Federated Learning via Hierarchical Self-Distillationcited by 2
  6. 2022CC-FedAvg: Computationally Customized Federated Averagingcited by 2
  7. 2024DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learningcited by 2
  8. 2026Toward Enhancing Representation Learning in Federated Multi-Task Settingscited by 2
  9. 2022Reducing Training Time in Cross-Silo Federated Learning using Multigraph Topologycited by 1
  10. 2023Breaking On-device Training Memory Wall: A Systematic Surveycited by 1
  11. 2023ECLM: Efficient Edge-Cloud Collaborative Learning with Continuous Environment Adaptationcited by 1
  12. 2024FedTrans: Efficient Federated Learning via Multi-Model Transformationcited by 1
  13. 2024Heterogeneity-Aware Coordination for Federated Learning via Stitching Pre-trained blockscited by 1
  14. 2025FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devicescited by 1
  15. 2025SFL-LEO: Asynchronous Split-Federated Learning Design for LEO Satellite-Ground Network Frameworkcited by 1
  16. 2026ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learningcited by 1
  17. 2023When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Frameworkcited by 0
  18. 2023Heterogeneous Federated Learning Using Knowledge Codistillationcited by 0
  19. 2024Federated Learning for Collaborative Inference Systems: The Case of Early Exit Networkscited by 0
  20. 2024MSfusion: A Dynamic Model Splitting Approach for Resource-Constrained Machines to Collaboratively Train Larger Modelscited by 0
  21. 2025FedOptima: Optimizing Resource Utilization in Federated Learningcited by 0
  22. 2026FEAST: Federated Shared-Space Training for Resource-Heterogeneous Clientscited by 0