mlsanityData to 14 Oct 2026

Why Go Full? Elevating Federated Learning Through Partial Network Updates

arXiv 2410.11559 ↗First posted 15 Oct 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated learning is a distributed machine learning paradigm designed to protect user data privacy, which has been successfully implemented across various scenarios. In traditional federated learning, the entire parameter set of local models is updated and averaged in each training round. Although this full network update method maximizes knowledge acquisition and sharing for each model layer, it prevents the layers of the global model from cooperating effectively to complete the tasks of each client, a challenge we refer to as layer mismatch. This mismatch problem recurs after every parameter averaging, consequently slowing down model convergence and degrading overall performance. To address the layer mismatch issue, we introduce the FedPart method, which restricts model updates to either a single layer or a few layers during each communication round. Furthermore, to maintain the efficiency of knowledge acquisition and sharing, we develop several strategies to select trainable layers in each round, including sequential updating and multi-round cycle training. Through both theoretical analysis and experiments, our findings demonstrate that the FedPart method significantly surpasses conventional full network update strategies in terms of convergence speed and accuracy, while also reducing communication and computational overheads.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

33 of 49 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2019Advances and Open Problems in Federated Learningcited by 3,046
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2019Deep Leakage from Gradientscited by 1,021
  11. 2021Model-Contrastive Federated Learningcited by 766
  12. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
21 more
  1. 2020The Future of Digital Health with Federated Learningcited by 633
  2. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  3. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  4. 2019Federated Learning with Personalization Layerscited by 541
  5. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 502
  6. 2021Towards Personalized Federated Learningcited by 464
  7. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  8. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  9. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  10. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  11. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  12. 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
  13. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
  14. 2018In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learningcited by 80
  15. 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
  16. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  17. 2024Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decompositioncited by 15
  18. 2021Partial Variable Training for Efficient On-Device Federated Learningcited by 14
  19. 2021Efficient and Private Federated Learning with Partially Trainable Networkscited by 13
  20. 2024Breaking the Memory Wall for Heterogeneous Federated Learning via Progressive Trainingcited by 10
  21. 2022CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantizationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2025Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunchcited by 28
  2. 2024Lion Cub: Minimizing Communication Overhead in Distributed Lioncited by 1
  3. 2024Learn More by Using Less: Distributed Learning with Energy-Constrained Devicescited by 0
  4. 2025UltraFlwr -- An Efficient Federated Surgical Object Detection Frameworkcited by 0
  5. 2025Federated Unsupervised Semantic Segmentationcited by 0
  6. 2026FedFrozen: Two-Stage Federated Optimization via Attention Kernel Freezingcited by 0