mlsanityData to 14 Oct 2026

Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering

arXiv 2205.11506 ↗First posted 23 May 2022cs.LG cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Federated learning is generally used in tasks where labels are readily available (e.g., next word prediction). Relaxing this constraint requires design of unsupervised learning techniques that can support desirable properties for federated training: robustness to statistical/systems heterogeneity, scalability with number of participants, and communication efficiency. Prior work on this topic has focused on directly extending centralized self-supervised learning techniques, which are not designed to have the properties listed above. To address this situation, we propose Orchestra, a novel unsupervised federated learning technique that exploits the federation's hierarchy to orchestrate a distributed clustering task and enforce a globally consistent partitioning of clients' data into discriminable clusters. We show the algorithmic pipeline in Orchestra guarantees good generalization performance under a linear probe, allowing it to outperform alternative techniques in a broad range of conditions, including variation in heterogeneity, number of clients, participation ratio, and local epochs.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 61 of its 90 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

61 of 90 references.

References · 61

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2019Advances and Open Problems in Federated Learningcited by 3,046
  7. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  12. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
38 more
  1. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  2. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  4. 2018Federated Learning with Non-IID Datacited by 1,108
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  6. 2020Adaptive Federated Optimizationcited by 1,046
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  10. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  11. 2017Federated Multi-Task Learningcited by 672
  12. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  13. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  14. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  15. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  16. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  17. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  18. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  19. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  20. 2019Fair Resource Allocation in Federated Learningcited by 394
  21. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 313
  22. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  23. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  24. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 256
  25. 2016Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clusteringcited by 237
  26. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  27. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 180
  28. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  29. 2018Tracking Emerges by Colorizing Videoscited by 166
  30. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  31. 2017Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimizationcited by 113
  32. 2020Online Deep Clustering for Unsupervised Representation Learningcited by 100
  33. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
  34. 2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
  35. 2020Federated Unsupervised Representation Learningcited by 93
  36. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  37. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  38. 2021On Large-Cohort Training for Federated Learningcited by 74

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 15; 2024: 13; 2025: 7; 2026: 5.

45 papers.

Sort:

  1. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
  2. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  3. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 41
  4. 2023Personalized Federated Learning under Mixture of Distributionscited by 37
  5. 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 30
  6. 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
  7. 2023Re-thinking Federated Active Learning based on Inter-class Diversitycited by 14
  8. 2023Federated Learning without Full Labels: A Surveycited by 13
  9. 2024Rethinking the Representation in Federated Unsupervised Learning with Non-IID Datacited by 13
  10. 2022The Future of Consumer Edge-AI Computingcited by 12
  11. 2023ProtoFL: Unsupervised Federated Learning via Prototypical Distillationcited by 12
  12. 2024Effective Heterogeneous Federated Learning via Efficient Hypernetwork-based Weight Generationcited by 9
33 more
  1. 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
  2. 2022Analyzing Data-Centric Properties for Graph Contrastive Learningcited by 7
  3. 2023Towards Unbiased Training in Federated Open-world Semi-supervised Learningcited by 7
  4. 2024A Mutual Information Perspective on Federated Contrastive Learningcited by 7
  5. 2024Navigating Distribution Shifts in Medical Image Analysis: A Surveycited by 5
  6. 2023Federated Continual Novel Class Learningcited by 4
  7. 2024Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Datacited by 4
  8. 2023FedIL: Federated Incremental Learning from Decentralized Unlabeled Data with Convergence Analysiscited by 3
  9. 2023FedMAE: Federated Self-Supervised Learning with One-Block Masked Auto-Encodercited by 3
  10. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  11. 2024Exploring Federated Self-Supervised Learning for General Purpose Audio Understandingcited by 2
  12. 2022Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learningcited by 1
  13. 2023Masked Autoencoders are Efficient Continual Federated Learnerscited by 1
  14. 2023UFed-GAN: A Secure Federated Learning Framework with Constrained Computation and Unlabeled Datacited by 1
  15. 2023Harnessing the Power of Federated Learning in Federated Contextual Banditscited by 1
  16. 2024Fair Federated Data Clustering through Personalization: Bridging the Gap between Diverse Data Distributionscited by 1
  17. 2024Jigsaw Game: Federated Clusteringcited by 1
  18. 2025FedDyMem: Efficient Federated Learning with Dynamic Memory and Memory-Reduce for Unsupervised Image Anomaly Detectioncited by 1
  19. 2025FSSUAVL: A Discriminative Framework using Vision Models for Federated Self-Supervised Audio and Image Understandingcited by 1
  20. 2026Understanding the Robustness of Distributed Self-Supervised Learning Frameworks Against Non-IID Datacited by 1
  21. 2022Federated Graph Representation Learning using Self-Supervisioncited by 0
  22. 2024Exploring Federated Deep Learning for Standardising Naming Conventions in Radiotherapy Datacited by 0
  23. 2024Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregationcited by 0
  24. 2024Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labelscited by 0
  25. 2024Federated Deep Subspace Clusteringcited by 0
  26. 2025Federated Unsupervised Semantic Segmentationcited by 0
  27. 2025Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learningcited by 0
  28. 2025FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computingcited by 0
  29. 2025Cooperative Pseudo Labeling for Unsupervised Federated Classificationcited by 0
  30. 2026Federated Hierarchical Clustering with Automatic Selection of Optimal Cluster Numberscited by 0
  31. 2026Semantic-based Distributed Learning for Diverse and Discriminative Representationscited by 0
  32. 2026Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysiscited by 0
  33. 2026StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splittingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Machine Unlearning of Federated Clusters
  2. 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillation
  3. 2026COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication
  4. 2024SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning
  5. 2024ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning
  6. 2024FedDW: Distilling Weights through Consistency Optimization in Heterogeneous Federated Learning
  7. 2022Communication-Efficient Federated Learning With Data and Client Heterogeneity
  8. 2021Cluster-driven Graph Federated Learning over Multiple Domains