Federated Learning with Flexible Architectures
Not on the map. Why?
Abstract (arXiv)
Traditional federated learning (FL) methods have limited support for clients with varying computational and communication abilities, leading to inefficiencies and potential inaccuracies in model training. This limitation hinders the widespread adoption of FL in diverse and resource-constrained environments, such as those with client devices ranging from powerful servers to mobile devices. To address this need, this paper introduces Federated Learning with Flexible Architectures (FedFA), an FL training algorithm that allows clients to train models of different widths and depths. Each client can select a network architecture suitable for its resources, with shallower and thinner networks requiring fewer computing resources for training. Unlike prior work in this area, FedFA incorporates the layer grafting technique to align clients' local architectures with the largest network architecture in the FL system during model aggregation. Layer grafting ensures that all client contributions are uniformly integrated into the global model, thereby minimizing the risk of any individual client's data skewing the model's parameters disproportionately and introducing security benefits. Moreover, FedFA introduces the scalable aggregation method to manage scale variations in weights among different network architectures. Experimentally, FedFA outperforms previous width and depth flexible aggregation strategies. Furthermore, FedFA demonstrates increased robustness against performance degradation in backdoor attack scenarios compared to earlier strategies.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 41 references that are in the map. First posted on arXiv 14 Jun 2024.
Builds on
27 of 41 references.
References · 27
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Searching for MobileNetV3cited by 1,984
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
- 2018How To Backdoor Federated Learningcited by 745
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
- 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
- 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
15 more
- 2020Threats to Federated Learning: A Surveycited by 171
- 2018Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?cited by 94
- 2019Machine Learning at the Network Edge: A Surveycited by 83
- 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
- 2020Towards Flexible Device Participation in Federated Learningcited by 64
- 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
- 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
- 2016Highway and Residual Networks learn Unrolled Iterative Estimationcited by 52
- 2023The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limitcited by 47
- 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
- 2020Communication-Efficient Federated Learning via Optimal Client Samplingcited by 39
- 2021Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneitycited by 27
- 2021Rethinking Skip Connection with Layer Normalization in Transformers and ResNetscited by 22
- 2022Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directionscited by 14
- 2022Sniper Backdoor: Single Client Targeted Backdoor Attack in Federated Learningcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.