mlsanityData to 7 Oct 2026

FedVeca: Federated Vectorized Averaging on Non-IID Data with Adaptive Bi-directional Global Objective

arXiv 2209.13803 ↗First posted 28 Sep 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a distributed machine learning framework to alleviate the data silos, where decentralized clients collaboratively learn a global model without sharing their private data. However, the clients' Non-Independent and Identically Distributed (Non-IID) data negatively affect the trained model, and clients with different numbers of local updates may cause significant gaps to the local gradients in each communication round. In this paper, we propose a Federated Vectorized Averaging (FedVeca) method to address the above problem on Non-IID data. Specifically, we set a novel objective for the global model which is related to the local gradients. The local gradient is defined as a bi-directional vector with step size and direction, where the step size is the number of local updates and the direction is divided into positive and negative according to our definition. In FedVeca, the direction is influenced by the step size, thus we average the bi-directional vectors to reduce the effect of different step sizes. Then, we theoretically analyze the relationship between the step sizes and the global objective, and obtain upper bounds on the step sizes per communication round. Based on the upper bounds, we design an algorithm for the server and the client to adaptively adjusts the step sizes that make the objective close to the optimum. Finally, we conduct experiments on different datasets, models and scenarios by building a prototype system, and the experimental results demonstrate the effectiveness and efficiency of the FedVeca method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

29 of 50 references.

References · 29

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  8. 2021Domain Generalization: A Surveycited by 760
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2017Federated Multi-Task Learningcited by 669
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2014Convex Optimization: Algorithms and Complexitycited by 569
17 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  3. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  4. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  5. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  6. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  7. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  8. 2019Federated Adversarial Domain Adaptationcited by 140
  9. 2018Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithmcited by 111
  10. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  11. 2018Tight Analyses for Non-Smooth Stochastic Gradient Descentcited by 94
  12. 2019Variance Reduced Local SGD with Lower Communication Complexitycited by 80
  13. 2019Variational Federated Multi-Task Learningcited by 69
  14. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  15. 2020Federated learning with class imbalance reductioncited by 52
  16. 2019Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptionscited by 48
  17. 2022Mitigating Data Heterogeneity in Federated Learning with Data Augmentationcited by 21

Built on by

Not cited yet.