mlsanityData to 7 Oct 2026

Fake It Till Make It: Federated Learning with Consensus-Oriented Generation

arXiv 2312.05966 ↗First posted 10 Dec 2023cs.LG cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

In federated learning (FL), data heterogeneity is one key bottleneck that causes model divergence and limits performance. Addressing this, existing methods often regard data heterogeneity as an inherent property and propose to mitigate its adverse effects by correcting models. In this paper, we seek to break this inherent property by generating data to complement the original dataset to fundamentally mitigate heterogeneity level. As a novel attempt from the perspective of data, we propose federated learning with consensus-oriented generation (FedCOG). FedCOG consists of two key components at the client side: complementary data generation, which generates data extracted from the shared global model to complement the original dataset, and knowledge-distillation-based model training, which distills knowledge from global model to local model based on the generated data to mitigate over-fitting the original heterogeneous dataset. FedCOG has two critical advantages: 1) it can be a plug-and-play module to further improve the performance of most existing FL methods, and 2) it is naturally compatible with standard FL protocols such as Secure Aggregation since it makes no modification in communication process. Extensive experiments on classical and real-world FL datasets show that FedCOG consistently outperforms state-of-the-art methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 52 of its 63 references that are in the map. First posted on arXiv 10 Dec 2023.

Builds on

52 of 63 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  8. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  9. 2018Federated Learning with Non-IID Datacited by 1,099
  10. 2020Adaptive Federated Optimizationcited by 1,036
  11. 2019Deep Leakage from Gradientscited by 1,004
  12. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
38 more
  1. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  2. 2021Model-Contrastive Federated Learningcited by 762
  3. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  4. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  5. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  6. 2020Federated Learning with Matched Averagingcited by 564
  7. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  8. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  9. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  10. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  11. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  12. 2019Fair Resource Allocation in Federated Learningcited by 389
  13. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  14. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  15. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  16. 2021A Field Guide to Federated Optimizationcited by 312
  17. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  18. 2018Applied Federated Learning: Improving Google Keyboard Query Suggestionscited by 202
  19. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
  20. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  21. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  22. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  23. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  24. 2021Federated Learning for Open Bankingcited by 138
  25. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
  26. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  27. 2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
  28. 2019Data-Free Adversarial Distillationcited by 90
  29. 2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 68
  30. 2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
  31. 2021LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learningcited by 63
  32. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
  33. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  34. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
  35. 2022Trading Off Privacy, Utility and Efficiency in Federated Learningcited by 33
  36. 2022Acceleration of Federated Learning with Alleviated Forgetting in Local Trainingcited by 29
  37. 2022FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learningcited by 27
  38. 2022FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentationcited by 24

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 3.

6 papers.

  1. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  2. 2024RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregationcited by 10
  3. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learningcited by 2
  4. 2025Incentivizing Inclusive Contributions in Model Sharing Marketscited by 1
  5. 2025Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregationcited by 1
  6. 2025Efficient Federated Learning with Encrypted Data Sharing for Data-Heterogeneous Edge Devicescited by 0