mlsanityData to 7 Oct 2026

FedCCL: Federated Dual-Clustered Feature Contrast Under Domain Heterogeneity

arXiv 2404.09259 ↗First posted 14 Apr 2024cs.CV cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) facilitates a privacy-preserving neural network training paradigm through collaboration between edge clients and a central server. One significant challenge is that the distributed data is not independently and identically distributed (non-IID), typically including both intra-domain and inter-domain heterogeneity. However, recent research is limited to simply using averaged signals as a form of regularization and only focusing on one aspect of these non-IID challenges. Given these limitations, this paper clarifies these two non-IID challenges and attempts to introduce cluster representation to address them from both local and global perspectives. Specifically, we propose a dual-clustered feature contrast-based FL framework with dual focuses. First, we employ clustering on the local representations of each client, aiming to capture intra-class information based on these local clusters at a high level of granularity. Then, we facilitate cross-client knowledge sharing by pulling the local representation closer to clusters shared by clients with similar semantics while pushing them away from clusters with dissimilar semantics. Second, since the sizes of local clusters belonging to the same class may differ for each client, we further utilize clustering on the global side and conduct averaging to create a consistent global signal for guiding each local training in a contrastive manner. Experimental results on multiple datasets demonstrate that our proposal achieves comparable or superior performance gain under intra-domain and inter-domain heterogeneity.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 58 of its 79 references that are in the map. First posted on arXiv 14 Apr 2024.

Builds on

58 of 79 references.

References · 58

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
38 more
  1. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  2. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  3. 2021Model-Contrastive Federated Learningcited by 762
  4. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  5. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 636
  6. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  7. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  8. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  9. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  10. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  11. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  12. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  13. 2019The many Shapley values for model explanationcited by 341
  14. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  15. 2021Federated Learning for Internet of Things: A Comprehensive Surveycited by 247
  16. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  17. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  18. 2018Adversarial Logit Pairingcited by 224
  19. 2021Adaptive Prototype Learning and Allocation for Few-Shot Segmentationcited by 207
  20. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  21. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  22. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
  23. 2020Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challengescited by 138
  24. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  25. 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 96
  26. 2021Contrastive Learning of Musical Representationscited by 81
  27. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  28. 2022Tackling Data Heterogeneity in Federated Learning with Class Prototypescited by 66
  29. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  30. 2017Hierarchical Clustering: Objective Functions and Algorithmscited by 60
  31. 2015Towards Universal Paraphrastic Sentence Embeddingscited by 59
  32. 2023PRIOR: Prototype Representation Joint Learning from Medical Images and Reportscited by 55
  33. 2023FedFed: Feature Distillation against Data Heterogeneity in Federated Learningcited by 54
  34. 2020Fair clustering via equitable group representationscited by 43
  35. 2018On the Robustness of the CVPR 2018 White-Box Adversarial Example Defensescited by 40
  36. 2021Video Contrastive Learning with Global Contextcited by 40
  37. 2021Node Selection Toward Faster Convergence for Federated Learning on Non-IID Datacited by 39
  38. 2022MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Seriescited by 28

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2025DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Surveycited by 6
  2. 2024Federated Hybrid Training and Self-Adversarial Distillation: Towards Robust Edge Networkscited by 2
  3. 2025Towards Communication-Efficient Adversarial Federated Learning for Robust Edge Intelligencecited by 1
  4. 2025FedFeat+: A Robust Federated Learning Framework Through Federated Aggregation and Differentially Private Feature-Based Classifier Retrainingcited by 1
  5. 2025Robust Federated Learning on Edge Devices with Domain Heterogeneitycited by 0