# FedSynth: Gradient Compression via Synthetic Data in Federated Learning

arXiv: https://arxiv.org/abs/2204.01273 · HTML: https://mlsanity.com/paper/2204.01273/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [signSGD: Compressed Optimisation for Non-Convex Problems](https://mlsanity.com/paper/1802.04434/) (2018)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [Asynchronous Federated Optimization](https://mlsanity.com/paper/1903.03934/) (2019)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations](https://mlsanity.com/paper/1906.02367/) (2019)
- [Federated Learning with Compression: Unified Analysis and Sharp Guarantees](https://mlsanity.com/paper/2007.01154/) (2020)
- [On Biased Compression for Distributed Learning](https://mlsanity.com/paper/2002.12410/) (2020)
- [Distributed Mean Estimation with Limited Communication](https://mlsanity.com/paper/1611.00429/) (2016)
- [Federated Learning via Synthetic Data](https://mlsanity.com/paper/2008.04489/) (2020)

## Built on by (most cited first, 30 of 31)

- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023)
- [Data Distillation: A Survey](https://mlsanity.com/paper/2301.04272/) (2023)
- [Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching](https://mlsanity.com/paper/2311.17950/) (2023)
- [Accelerating Dataset Distillation via Model Augmentation](https://mlsanity.com/paper/2212.06152/) (2022)
- [A Survey on Dataset Distillation: Approaches, Applications and Future Directions](https://mlsanity.com/paper/2305.01975/) (2023)
- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Backdoor Attacks Against Dataset Distillation](https://mlsanity.com/paper/2301.01197/) (2023)
- [A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency](https://mlsanity.com/paper/2307.10655/) (2023)
- [Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents](https://mlsanity.com/paper/2312.01537/) (2023)
- [Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation](https://mlsanity.com/paper/2303.01687/) (2023)
- [Can pre-trained models assist in dataset distillation?](https://mlsanity.com/paper/2310.03295/) (2023)
- [Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2405.07925/) (2024)
- [Federated Learning on Virtual Heterogeneous Data with Local-global Distillation](https://mlsanity.com/paper/2303.02278/) (2023)
- [Communication-efficient Federated Learning with Single-Step Synthetic Features Compressor for Faster Convergence](https://mlsanity.com/paper/2302.13562/) (2023)
- [Embarassingly Simple Dataset Distillation](https://mlsanity.com/paper/2311.07025/) (2023)
- [A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentation](https://mlsanity.com/paper/2306.08720/) (2023)
- [Tackling Data Heterogeneity in Federated Time Series Forecasting](https://mlsanity.com/paper/2411.15716/) (2024)
- [E-3SFC: Communication-Efficient Federated Learning with Double-way Features Synthesizing](https://mlsanity.com/paper/2502.03092/) (2025)
- [Exemplar-condensed Federated Class-incremental Learning](https://mlsanity.com/paper/2412.18926/) (2024)
- [C^2Prompt: Class-aware Client Knowledge Interaction for Federated Continual Learning](https://mlsanity.com/paper/2509.19674/) (2025)
- [One-Shot Federated Learning with Bayesian Pseudocoresets](https://mlsanity.com/paper/2406.02177/) (2024)
- [Breaking Resource Barriers in Speech Emotion Recognition via Data Distillation](https://mlsanity.com/paper/2406.15119/) (2024)
- [One-Shot Collaborative Data Distillation](https://mlsanity.com/paper/2408.02266/) (2024)
- [Using Synthetic Data to Mitigate Unfairness and Preserve Privacy in Collaborative Machine Learning](https://mlsanity.com/paper/2409.09532/) (2024)
- [Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching](https://mlsanity.com/paper/2305.18076/) (2023)
- [Federated Learning Optimization: A Comparative Study of Data and Model Exchange Strategies in Dynamic Networks](https://mlsanity.com/paper/2406.10798/) (2024)
- [UDD: Dataset Distillation via Mining Underutilized Regions](https://mlsanity.com/paper/2408.16268/) (2024)
- [WarmFed: Federated Learning with Warm-Start for Globalization and Personalization Via Personalized Diffusion Models](https://mlsanity.com/paper/2503.03110/) (2025)
- [Concordia: Self-Improving Synthetic Tables for Federated LLMs](https://mlsanity.com/paper/2605.09855/) (2026)

## Abstract (arXiv)

Model compression is important in federated learning (FL) with large models to reduce communication cost. Prior works have been focusing on sparsification based compression that could desparately affect the global model accuracy. In this work, we propose a new scheme for upstream communication where instead of transmitting the model update, each client learns and transmits a light-weight synthetic dataset such that using it as the training data, the model performs similarly well on the real training data. The server will recover the local model update via the synthetic data and apply standard aggregation. We then provide a new algorithm FedSynth to learn the synthetic data locally. Empirically, we find our method is comparable/better than random masking baselines in all three common federated learning benchmark datasets.