mlsanityData to 14 Oct 2026

Fed-CO2: Cooperation of Online and Offline Models for Severe Data Heterogeneity in Federated Learning

arXiv 2312.13923 ↗First posted 21 Dec 2023cs.LG cs.AI cs.DCcited by 7

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has emerged as a promising distributed learning paradigm that enables multiple clients to learn a global model collaboratively without sharing their private data. However, the effectiveness of FL is highly dependent on the quality of the data that is being used for training. In particular, data heterogeneity issues, such as label distribution skew and feature skew, can significantly impact the performance of FL. Previous studies in FL have primarily focused on addressing label distribution skew data heterogeneity, while only a few recent works have made initial progress in tackling feature skew issues. Notably, these two forms of data heterogeneity have been studied separately and have not been well explored within a unified FL framework. To address this gap, we propose Fed-CO2, a universal FL framework that handles both label distribution skew and feature skew within a Cooperation mechanism between the Online and Offline models. Specifically, the online model learns general knowledge that is shared among all clients, while the offline model is trained locally to learn the specialized knowledge of each individual client. To further enhance model cooperation in the presence of feature shifts, we design an intra-client knowledge transfer mechanism that reinforces mutual learning between the online and offline models, and an inter-client knowledge transfer mechanism to increase the models' domain generalization ability. Extensive experiments show that our Fed-CO2 outperforms a wide range of existing personalized federated learning algorithms in terms of handling label distribution skew and feature skew, both individually and collectively. The empirical results are supported by our convergence analyses in a simplified setting.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

36 of 50 references.

References · 36

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  6. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  7. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  8. 2020Adaptive Federated Optimizationcited by 1,046
  9. 2016HyperNetworkscited by 1,027
  10. 2021Model-Contrastive Federated Learningcited by 766
  11. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  12. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
24 more
  1. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  2. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  3. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  4. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  5. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
  6. 2019Federated Learning with Personalization Layerscited by 541
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  9. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  10. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  11. 2021Personalized Federated Learning using Hypernetworkscited by 212
  12. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  13. 2019Federated Evaluation of On-device Personalizationcited by 147
  14. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  15. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
  16. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  17. 2021Personalized Federated Learning with Gaussian Processescited by 76
  18. 2020Federated Mutual Learningcited by 71
  19. 2021Personalized Federated Learning through Local Memorizationcited by 57
  20. 2022FedTP: Federated Learning by Transformer Personalizationcited by 36
  21. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  22. 2021Personalized Federated Learning with Adaptive Batchnorm for Healthcarecited by 26
  23. 2023The Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillationcited by 25
  24. 2020Optimization Theory for ReLU Neural Networks Trained with Normalization Layerscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 2; 2026: 1.

7 papers.

  1. 2024Harmonizing Generalization and Personalization in Federated Prompt Learningcited by 27
  2. 2024Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Methodcited by 22
  3. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 17
  4. 2024FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learningcited by 2
  5. 2025Federated Cross-Modal Style-Aware Prompt Generationcited by 0
  6. 2025FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Promptscited by 0
  7. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0