mlsanityData to 7 Oct 2026

An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning

arXiv 2306.12088 ↗First posted 21 Jun 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Communication overhead is one of the major challenges in Federated Learning(FL). A few classical schemes assume the server can extract the auxiliary information about training data of the participants from the local models to construct a central dummy dataset. The server uses the dummy dataset to finetune aggregated global model to achieve the target test accuracy in fewer communication rounds. In this paper, we summarize the above solutions into a data-based communication-efficient FL framework. The key of the proposed framework is to design an efficient extraction module(EM) which ensures the dummy dataset has a positive effect on finetuning aggregated global model. Different from the existing methods that use generator to design EM, our proposed method, FedINIBoost borrows the idea of gradient match to construct EM. Specifically, FedINIBoost builds a proxy dataset of the real dataset in two steps for each participant at each communication round. Then the server aggregates all the proxy datasets to form a central dummy dataset, which is used to finetune aggregated global model. Extensive experiments verify the superiority of our method compared with the existing classical method, FedAVG, FedProx, Moon and FedFTG. Moreover, FedINIBoost plays a significant role in finetuning the performance of aggregated global model at the initial stage of FL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

18 of 20 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2014Conditional Generative Adversarial Netscited by 2,152
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  9. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  11. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  12. 2021Federated Learning Based on Dynamic Regularizationcited by 555
6 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  3. 2018Sparsified SGD with Memorycited by 373
  4. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  5. 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
  6. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140

Built on by

Not cited yet.