Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning

arXiv 2503.06457 ↗First posted 9 Mar 2025cs.CV cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Data heterogeneity in federated learning, characterized by a significant misalignment between local and global distributions, leads to divergent local optimization directions and hinders global model training. Existing studies mainly focus on optimizing local updates or global aggregation, but these indirect approaches demonstrate instability when handling highly heterogeneous data distributions, especially in scenarios where label skew and domain skew coexist. To address this, we propose a geometry-guided data generation method that centers on simulating the global embedding distribution locally. We first introduce the concept of the geometric shape of an embedding distribution and then address the challenge of obtaining global geometric shapes under privacy constraints. Subsequently, we propose GGEUR, which leverages global geometric shapes to guide the generation of new samples, enabling a closer approximation to the ideal global distribution. In single-domain scenarios, we augment samples based on global geometric shapes to enhance model generalization; in multi-domain scenarios, we further employ class prototypes to simulate the global distribution across domains. Extensive experimental results demonstrate that our method significantly enhances the performance of existing approaches in handling highly heterogeneous data, including scenarios with label skew, domain skew, and their coexistence. Code published at: https://github.com/WeiDai-David/2025CVPR_GGEUR

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 47 of its 70 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

47 of 70 references.

References · 47

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
2018Federated Optimization in Heterogeneous Networkscited by 2,971
2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
2018Federated Learning with Non-IID Datacited by 1,099
2020Adaptive Federated Optimizationcited by 1,036
2017Deeper, Broader and Artier Domain Generalizationcited by 983
2021Model-Contrastive Federated Learningcited by 762
2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565

35 more

2020Federated Learning with Matched Averagingcited by 564
2021Federated Learning Based on Dynamic Regularizationcited by 555
2019Fair Resource Allocation in Federated Learningcited by 389
2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
2019Model Fusion via Optimal Transportcited by 217
2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
2020FedBE: Making Bayesian Model Ensemble Applicable to Federated Learningcited by 176
2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 174
2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
2023FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learningcited by 70
2021Towards Fair Federated Learning with Zero-Shot Data Augmentationcited by 64
2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 57
2023No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifiercited by 55
2018Effects of Degradations on Deep Neural Network Architecturescited by 47
2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
2021FedCG: Leverage Conditional GAN for Protecting Privacy and Maintaining Competitive Performance in Federated Learningcited by 39
2022Multi-Level Branched Regularization for Federated Learningcited by 34
2024Fair Federated Learning under Domain Skew with Local Consistency and Domain Diversitycited by 28
2022Towards Federated Learning against Noisy Labels via Local Self-Regularizationcited by 26
2022Delving into Semantic Scale Imbalancecited by 22
2023FedFA: Federated Feature Augmentationcited by 22
2021Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentationcited by 19
2022FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classificationcited by 18
2024Geometric Prior Guided Feature Representation Learning for Long-Tailed Classificationcited by 10
2022On the effectiveness of partial variance reduction in federated learning with heterogeneous datacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.