# CCFC++: Enhancing Federated Clustering through Feature Decorrelation

arXiv: https://arxiv.org/abs/2402.12852 · HTML: https://mlsanity.com/paper/2402.12852/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization](https://mlsanity.com/paper/1802.06509/) (2018)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Gradient descent aligns the layers of deep linear networks](https://mlsanity.com/paper/1810.02032/) (2018)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Multi-Center Federated Learning: Clients Clustering for Better Personalization](https://mlsanity.com/paper/2108.08647/) (2021)
- [Heterogeneity for the Win: One-Shot Federated Clustering](https://mlsanity.com/paper/2103.00697/) (2021)
- [A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions](https://mlsanity.com/paper/2206.07579/) (2022)
- [Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation](https://mlsanity.com/paper/2106.00131/) (2021)
- [Client Selection in Federated Learning: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2211.01549/) (2022)
- [Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning](https://mlsanity.com/paper/2112.04731/) (2021)
- [Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)](https://mlsanity.com/paper/2201.07316/) (2022)
- [Evaluation of Community Detection Methods](https://mlsanity.com/paper/1807.01130/) (2018)
- [Privacy-Preserving Federated Deep Clustering based on GAN](https://mlsanity.com/paper/2211.16965/) (2022)
- [CCFC: Bridging Federated Clustering and Contrastive Learning](https://mlsanity.com/paper/2401.06634/) (2024)
- [Federated clustering with GAN-based data synthesis](https://mlsanity.com/paper/2210.16524/) (2022)

## Built on by (most cited first, 3 of 3)

- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Fair Federated Data Clustering through Personalization: Bridging the Gap between Diverse Data Distributions](https://mlsanity.com/paper/2407.04302/) (2024)
- [OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction](https://mlsanity.com/paper/2505.13071/) (2025)

## Abstract (arXiv)

In federated clustering, multiple data-holding clients collaboratively group data without exchanging raw data. This field has seen notable advancements through its marriage with contrastive learning, exemplified by Cluster-Contrastive Federated Clustering (CCFC). However, CCFC suffers from heterogeneous data across clients, leading to poor and unrobust performance. Our study conducts both empirical and theoretical analyses to understand the impact of heterogeneous data on CCFC. Findings indicate that increased data heterogeneity exacerbates dimensional collapse in CCFC, evidenced by increased correlations across multiple dimensions of the learned representations. To address this, we introduce a decorrelation regularizer to CCFC. Benefiting from the regularizer, the improved method effectively mitigates the detrimental effects of data heterogeneity, and achieves superior performance, as evidenced by a marked increase in NMI scores, with the gain reaching as high as 0.32 in the most pronounced case.