mlsanityData to 7 Oct 2026

FedSynth: Gradient Compression via Synthetic Data in Federated Learning

arXiv 2204.01273 ↗First posted 4 Apr 2022cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Model compression is important in federated learning (FL) with large models to reduce communication cost. Prior works have been focusing on sparsification based compression that could desparately affect the global model accuracy. In this work, we propose a new scheme for upstream communication where instead of transmitting the model update, each client learns and transmits a light-weight synthetic dataset such that using it as the training data, the model performs similarly well on the real training data. The server will recover the local model update via the synthetic data and apply standard aggregation. We then provide a new algorithm FedSynth to learn the synthetic data locally. Empirically, we find our method is comparable/better than random masking baselines in all three common federated learning benchmark datasets.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 14 of its 20 references that are in the map. First posted on arXiv 4 Apr 2022.

Builds on

14 of 20 references.

References · 14

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2018LEAF: A Benchmark for Federated Settingscited by 780
  3. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  4. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  5. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  6. 2018Model compression via distillation and quantizationcited by 287
  7. 2019Asynchronous Federated Optimizationcited by 264
  8. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  9. 2018Dataset Distillationcited by 224
  10. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  11. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  12. 2020On Biased Compression for Distributed Learningcited by 140
2 more
  1. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  2. 2020Federated Learning via Synthetic Datacited by 53

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 15; 2024: 9; 2025: 4; 2026: 1.

30 papers.

Sort:

By area:

  1. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  2. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  3. 2023Data Distillation: A Surveycited by 86
  4. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  5. 2022Accelerating Dataset Distillation via Model Augmentationcited by 62
  6. 2023A Survey on Dataset Distillation: Approaches, Applications and Future Directionscited by 32
  7. 2025The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutionscited by 24
  8. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiencycited by 21
  9. 2023Backdoor Attacks Against Dataset Distillationcited by 20
  10. 2023Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latentscited by 16
  11. 2023Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generationcited by 15
  12. 2023Can pre-trained models assist in dataset distillation?cited by 12
18 more
  1. 2024Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Datacited by 10
  2. 2023Federated Learning on Virtual Heterogeneous Data with Local-global Distillationcited by 6
  3. 2023Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergencecited by 5
  4. 2023Embarassingly Simple Dataset Distillationcited by 5
  5. 2023A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentationcited by 4
  6. 2025E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizingcited by 3
  7. 2024Tackling Data Heterogeneity in Federated Time Series Forecastingcited by 2
  8. 2024Exemplar-condensed Federated Class-incremental Learningcited by 2
  9. 2025C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learningcited by 2
  10. 2024One-Shot Federated Learning with Bayesian Pseudocoresetscited by 1
  11. 2024Breaking Resource Barriers in Speech Emotion Recognition via Data Distillationcited by 1
  12. 2024One-Shot Collaborative Data Distillationcited by 1
  13. 2023Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matchingcited by 0
  14. 2024Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networkscited by 0
  15. 2024UDD: Dataset Distillation via Mining Underutilized Regionscited by 0
  16. 2024Using Synthetic Data to Mitigate Unfairness and Preserve Privacy in Collaborative Machine Learningcited by 0
  17. 2025WarmFed: Federated Learning with Warm-Start for Globalization and Personalization Via Personalized Diffusion Modelscited by 0
  18. 2026Concordia: Self-Improving Synthetic Tables for Federated LLMscited by 0