mlsanityData to 7 Oct 2026

Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data

arXiv 2403.16398 ↗First posted 25 Mar 2024cs.LG cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Federated learning achieves effective performance in modeling decentralized data. In practice, client data are not well-labeled, which makes it potential for federated unsupervised learning (FUSL) with non-IID data. However, the performance of existing FUSL methods suffers from insufficient representations, i.e., (1) representation collapse entanglement among local and global models, and (2) inconsistent representation spaces among local models. The former indicates that representation collapse in local model will subsequently impact the global model and other local models. The latter means that clients model data representation with inconsistent parameters due to the deficiency of supervision signals. In this work, we propose FedU2 which enhances generating uniform and unified representation in FUSL with non-IID data. Specifically, FedU2 consists of flexible uniform regularizer (FUR) and efficient unified aggregator (EUA). FUR in each client avoids representation collapse via dispersing samples uniformly, and EUA in server promotes unified representation by constraining consistent client model updating. To extensively validate the performance of FedU2, we conduct both cross-device and cross-silo evaluation experiments on two benchmark datasets, i.e., CIFAR10 and CIFAR100.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

29 of 43 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  8. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  9. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  10. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  11. 2020Understanding the Behaviour of Contrastive Losscited by 397
  12. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
17 more
  1. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  2. 2020Revisiting Training Strategies and Generalization Performance in Deep Metric Learningcited by 99
  3. 2020Federated Unsupervised Representation Learningcited by 92
  4. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  5. 2022Divergence-aware Federated Self-Supervised Learningcited by 71
  6. 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 68
  7. 2019Sinkhorn Divergences for Unbalanced Optimal Transportcited by 52
  8. 2020Federated Learning Meets Multi-objective Optimizationcited by 50
  9. 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
  10. 2022Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learningcited by 45
  11. 2022Understanding Collapse in Non-Contrastive Siamese Representation Learningcited by 42
  12. 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillationcited by 40
  13. 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 29
  14. 2023VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distributioncited by 17
  15. 2023Federated Learning without Full Labels: A Surveycited by 12
  16. 2023ProtoFL: Unsupervised Federated Learning via Prototypical Distillationcited by 11
  17. 2023HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learningcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 3.

12 papers.

  1. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  2. 2025You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Datacited by 5
  3. 2025FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Modelscited by 2
  4. 2025Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentationcited by 1
  5. 2026Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Datacited by 1
  6. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 0
  7. 2025Federated Unsupervised Semantic Segmentationcited by 0
  8. 2025Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learningcited by 0
  9. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  10. 2025FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurementscited by 0
  11. 2026Semantic-based Distributed Learning for Diverse and Discriminative Representationscited by 0
  12. 2026Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysiscited by 0