mlsanityData to 14 Oct 2026

FedEGG: Federated Learning with Explicit Global Guidance

arXiv 2404.11888 ↗First posted 18 Apr 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) holds great potential for diverse applications owing to its privacy-preserving nature. However, its convergence is often challenged by non-IID data distributions, limiting its effectiveness in real-world deployments. Existing methods help address these challenges via optimization-based client constraints, adaptive client selection, or the use of pre-trained models or synthetic data. In this work, we reinterpret these approaches as all introducing an implicit guiding task to regularize and steer client learning. Following this insight, we propose to introduce an explicit global guiding task into the current FL framework to improve convergence and performance. To this end, we present FedEGG, a new FL algorithm that constructs a global guiding task using a well-defined, easy-to-converge learning task based on a public dataset and Large Language Models (LLMs). This approach effectively combines the strengths of federated (the original FL task) and centralized (the global guiding task) learning. We provide a theoretical analysis of FedEGG's convergence, examining the impact of data heterogeneity between the guiding and FL tasks and the guiding strength. Our analysis derives an upper bound for the optimal guiding strength, offering practical insights for implementation. Empirically, FedEGG demonstrates superior performance over state-of-the-art FL methods under both IID and non-IID settings, and further improves their performances when combined.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 53 references that are in the map. First posted on arXiv 18 Apr 2024.

Builds on

45 of 53 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  6. 2019Advances and Open Problems in Federated Learningcited by 3,046
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  8. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  9. 2022Visual Prompt Tuningcited by 1,681
  10. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  12. 2018Federated Learning with Non-IID Datacited by 1,108
33 more
  1. 2020Adaptive Federated Optimizationcited by 1,046
  2. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  3. 2020Federated Learning with Matched Averagingcited by 564
  4. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  5. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  6. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  7. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  8. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  9. 2019SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentumcited by 121
  10. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  11. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 101
  12. 2020TiFL: A Tier-based Federated Learning Systemcited by 96
  13. 2019Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation Systemcited by 79
  14. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  15. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  16. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  17. 2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62
  18. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  19. 2020Faster Non-Convex Federated Learning via Global and Local Momentumcited by 48
  20. 2021STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learningcited by 43
  21. 2020Communication-Efficient Federated Learning via Optimal Client Samplingcited by 39
  22. 2023Momentum Benefits Non-IID Federated Learning Simply and Provablycited by 38
  23. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  24. 2024DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learningcited by 32
  25. 2021Local Adaptivity in Federated Learning: Convergence and Consistencycited by 31
  26. 2020Bandit-based Communication-Efficient Client Selection Strategies for Federated Learningcited by 29
  27. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  28. 2021FedFace: Collaborative Learning of Face Recognition Modelcited by 26
  29. 2023Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learningcited by 20
  30. 2023Solving a Class of Non-Convex Minimax Optimization in Federated Learningcited by 10
  31. 2020Accurate and Fast Federated Learning via Combinatorial Multi-Armed Banditscited by 7
  32. 2023FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimizationcited by 7
  33. 2021On Addressing Heterogeneity in Federated Learning for Autonomous Vehicles Connected to a Drone Orchestratorcited by 6

Built on by

Not cited yet.