mlsanityData to 7 Oct 2026

FedImpro: Measuring and Improving Client Update in Federated Learning

arXiv 2402.07011 ↗First posted 10 Feb 2024cs.LG cs.AI cs.DCcited by 15

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) models often experience client drift caused by heterogeneous data, where the distribution of data differs across clients. To address this issue, advanced research primarily focuses on manipulating the existing gradients to achieve more consistent client models. In this paper, we present an alternative perspective on client drift and aim to mitigate it by generating improved local models. First, we analyze the generalization contribution of local training and conclude that this generalization contribution is bounded by the conditional Wasserstein distance between the data distribution of different clients. Then, we propose FedImpro, to construct similar conditional distributions for local training. Specifically, FedImpro decouples the model into high-level and low-level components, and trains the high-level portion on reconstructed feature distributions. This approach enhances the generalization contribution and reduces the dissimilarity of gradients in FL. Experimental results show that FedImpro can help FL defend against data heterogeneity and enhance the generalization performance of the model.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 81 of its 111 references that are in the map. First posted on arXiv 10 Feb 2024.

Builds on

81 of 111 references.

References · 81

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  9. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  10. 2018Federated Learning with Non-IID Datacited by 1,099
  11. 2020Adaptive Federated Optimizationcited by 1,036
  12. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
38 more
  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  2. 2018LEAF: A Benchmark for Federated Settingscited by 780
  3. 2021Model-Contrastive Federated Learningcited by 762
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  6. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  8. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  9. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  10. 2021Towards Personalized Federated Learningcited by 462
  11. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  12. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 371
  13. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  14. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  15. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  16. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  17. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  18. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  19. 2021GFlowNet Foundationscited by 286
  20. 2018Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Datacited by 242
  21. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  22. 2019Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptationcited by 235
  23. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  24. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  25. 2020Minibatch vs Local SGD for Heterogeneous Distributed Learningcited by 147
  26. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 135
  27. 2019Training Neural Networks with Local Error Signalscited by 135
  28. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  29. 2020Federated Visual Classification with Real-World Data Distributioncited by 126
  30. 2020What makes instance discrimination good for transfer learning?cited by 111
  31. 2018Large Margin Deep Networks for Classificationcited by 110
  32. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  33. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  34. 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
  35. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
  36. 2019VC Classes are Adversarially Robustly Learnable, but Only Improperlycited by 83
  37. 2020Regularizing Deep Networks with Semantic Data Augmentationcited by 73
  38. 2021On Large-Cohort Training for Federated Learningcited by 73

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 5; 2026: 2.

15 papers.

Sort:

  1. 2024LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuningcited by 88
  2. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  3. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  4. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  5. 2024FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compressioncited by 7
  6. 2025Federated Domain Generalization with Data-free On-server Matching Gradientcited by 7
  7. 2024Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learningcited by 4
  8. 2024GAS: Generative Activation-Aided Asynchronous Split Federated Learningcited by 3
  9. 2025FedWSIDD: Federated Whole Slide Image Classification via Dataset Distillationcited by 3
  10. 2025Optimized Local Updates in Federated Learning via Reinforcement Learningcited by 1
  11. 2024FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuningcited by 0
  12. 2025Provably Near-Optimal Federated Ensemble Distillation with Negligible Overheadcited by 0
3 more
  1. 2025Data Valuation and Selection in a Federated Model Marketplacecited by 0
  2. 2026Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integrationcited by 0
  3. 2026Latent Information Sharing for Accelerating Federated Learningcited by 0