Bridging Generalization Gap of Heterogeneous Federated Clients Using Generative Models
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) is a privacy-preserving machine learning framework facilitating collaborative training across distributed clients. However, its performance is often compromised by data heterogeneity among participants, which can result in local models with limited generalization capability. Traditional model-homogeneous approaches address this issue primarily by regularizing local training procedures or dynamically adjusting client weights during aggregation. Nevertheless, these methods become unsuitable in scenarios involving clients with heterogeneous model architectures. In this paper, we propose a model-heterogeneous FL framework that enhances clients' generalization performance on unseen data without relying on parameter aggregation. Instead of model parameters, clients share feature distribution statistics (mean and covariance) with the server. Then each client trains a variational transposed convolutional neural network using Gaussian latent variables sampled from these distributions, and use it to generate synthetic data. By fine-tuning local models with the synthetic data, clients achieve significant improvement of generalization ability. Experimental results demonstrate that our approach not only attains higher generalization accuracy compared to existing model-heterogeneous FL frameworks, but also reduces communication costs and memory consumption.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 3 Aug 2025.
Builds on
45 of 62 references.
References · 45
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2022Diffusion Models in Vision: A Surveycited by 1,015
- 2017Federated Multi-Task Learningcited by 672
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
- 2020Personalized Federated Learning with Moreau Envelopescited by 618
- 2021Federated Learning Based on Dynamic Regularizationcited by 562
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
33 more
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
- 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 346
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
- 2019Robust Aggregation for Federated Learningcited by 317
- 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
- 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
- 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
- 2016A guide to convolution arithmetic for deep learningcited by 185
- 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
- 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
- 2023Improved Distribution Matching for Dataset Condensationcited by 124
- 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
- 2020Learning to Detect Malicious Clients for Robust Federated Learningcited by 88
- 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 69
- 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
- 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
- 2020Federated Model Distillation with Noise-Free Differential Privacycited by 40
- 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
- 2021FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Modelscited by 31
- 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
- 2023Phoenix: A Federated Generative Diffusion Modelcited by 22
- 2024Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchorscited by 21
- 2024Personalized Federated Learning via Feature Distribution Adaptationcited by 13
- 2024Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learningcited by 11
- 2022Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusioncited by 10
- 2023FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategycited by 10
- 2021RobustFed: A Truth Inference Approach for Robust Federated Learningcited by 9
- 2023CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Datacited by 9
- 2023Aggregating Capacity in FL through Successive Layer Training for Computationally-Constrained Devicescited by 7
- 2025Federated Learning for Diffusion Modelscited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.