Federated Unsupervised Representation Learning
Not on the map. Why?
Abstract (arXiv)
To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without the unified information among clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose Federated Constrastive Averaging with dictionary and alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive loss for local model training. Through extensive experiments with three evaluation protocols in IID and Non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.
Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 11; 2022: 28; 2023: 16; 2024: 19; 2025: 11; 2026: 7.
92 papers.
Sort:
By area:
- 2021Model-Contrastive Federated Learningcited by 762
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
- 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
- 2022Divergence-aware Federated Self-Supervised Learningcited by 71
- 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 68
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
- 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
- 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
- 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
- 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillationcited by 40
- 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
38 more
- 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
- 2022Federated Contrastive Learning for Volumetric Medical Image Segmentationcited by 32
- 2022On the Convergence of Clustered Federated Learningcited by 31
- 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 29
- 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domainscited by 27
- 2021SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervisioncited by 26
- 2023Federated Generalized Category Discoverycited by 26
- 2021Joint Optimization in Edge-Cloud Continuum for Federated Unsupervised Person Re-identificationcited by 20
- 2022Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phasescited by 18
- 2022A Hybrid Self-Supervised Learning Framework for Vertical Federated Learningcited by 17
- 2021Collaborative Semantic Aggregation and Calibration for Federated Domain Generalizationcited by 16
- 2022Distributed Contrastive Learning for Medical Image Segmentationcited by 16
- 2021Decentralized Unsupervised Learning of Visual Representationscited by 15
- 2023Universal Domain Adaptation via Compressive Attention Matchingcited by 15
- 2022Federated Self-supervised Learning for Video Understandingcited by 13
- 2022Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Datacited by 12
- 2022Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?cited by 12
- 2023Federated Learning without Full Labels: A Surveycited by 12
- 2024Rethinking the Representation in Federated Unsupervised Learning with Non-IID Datacited by 12
- 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
- 2022FeDXL: Provable Federated Learning for Deep X-Risk Optimizationcited by 10
- 2022Federated Continual Learning for Text Classification via Selective Inter-client Transfercited by 9
- 2022FLAME: Federated Learning Across Multi-device Environmentscited by 8
- 2022Anomaly Detection through Unsupervised Federated Learningcited by 8
- 2023Federated Learning for Large-Scale Scene Modeling with Neural Radiance Fieldscited by 8
- 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applicationscited by 8
- 2024Distributionally Generative Augmentation for Fair Facial Attribute Classificationcited by 8
- 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
- 2024A Mutual Information Perspective on Federated Contrastive Learningcited by 7
- 2024Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradientscited by 7
- 2022Federated Momentum Contrastive Clusteringcited by 6
- 2022Self-supervised On-device Federated Learning from Unlabeled Streamscited by 6
- 2022Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchangecited by 5
- 2023Contrastive encoder pre-training-based clustered federated learning for heterogeneous datacited by 5
- 2024FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Datacited by 5
- 2023Federated Continual Novel Class Learningcited by 4
- 2024Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Datacited by 4
- 2021Towards Communication-Efficient and Privacy-Preserving Federated Representation Learningcited by 3
50 most cited of 92 All 92 (JSON)
- 2026Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data18 Sep 2026
- 2026Federated Attention Autoencoders with a Stochastic Aggregation Scheme for Anomaly Detection9 Aug 2026
- 2026Sheaf-Based Federated Representation Learning8 Aug 2026
- 2026StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting26 May 2026
- 2026Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networks25 May 2026
- 2026Semantic-based Distributed Learning for Diverse and Discriminative Representations20 Apr 2026
- 2026FedCova: Robust Federated Covariance Learning Against Noisy Labels4 Mar 2026
- 2025Local K-Similarity Constraint for Federated Learning with Label Noise9 Nov 2025
- 2025FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements14 Oct 2025
- 2025Parameter-Efficient and Personalized Federated Training of Generative Models at the Edge11 Oct 2025
- 2025Cooperative Pseudo Labeling for Unsupervised Federated Classification11 Oct 2025
- 2025IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learning11 Aug 2025
38 more
- 2025Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learning2 Aug 2025
- 2025FORLA: Federated Object-centric Representation Learning with Slot Attention3 Jun 2025
- 2025LocalKMeans: Convergence of Lloyd's Algorithm with Distributed Local Iterations23 May 2025
- 2025Federated Learning with Unlabeled Clients: Personalization Can Happen in Low Dimensions21 May 2025
- 2025Approximated Behavioral Metric-based State Projection for Federated Reinforcement Learning15 May 2025
- 2025Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentation30 Mar 2025
- 2024Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data18 Dec 2024
- 2024Federated Contrastive Learning of Graph-Level Representations18 Nov 2024
- 2024Bridging Gaps: Federated Multi-View Clustering in Heterogeneous Hybrid Views12 Oct 2024
- 2024Contrastive Federated Learning with Tabular Data Silos10 Sep 2024
- 2024Jigsaw Game: Federated Clustering17 Jul 2024
- 2024Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregation14 Jul 2024
- 2024A deep cut into Split Federated Self-supervised Learning12 Jun 2024
- 2024Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients25 May 2024
- 2024Federated Learning for Misbehaviour Detection with Variational Autoencoders and Gaussian Mixture Models16 May 2024
- 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations16 May 2024
- 2024FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data7 May 2024
- 2024A Mutual Information Perspective on Federated Contrastive Learning3 May 2024
- 2024Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labels15 Apr 2024
- 2024Distributionally Robust Alignment for Medical Federated Vision-Language Pre-training Under Data Heterogeneity5 Apr 2024
- 2024Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data25 Mar 2024
- 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains14 Mar 2024
- 2024Distributionally Generative Augmentation for Fair Facial Attribute Classification11 Mar 2024
- 2024A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications3 Mar 2024
- 2024Resource-efficient Layer-wise Federated Self-supervised Learning22 Jan 2024
- 2023Harnessing the Power of Federated Learning in Federated Contextual Bandits26 Dec 2023
- 2023Federated Continual Novel Class Learning21 Dec 2023
- 2023A Framework for Exploring Federated Community Detection14 Dec 2023
- 2023Contrastive encoder pre-training-based clustered federated learning for heterogeneous data28 Nov 2023
- 2023Federated Self-Supervised Learning of Monocular Depth Estimators for Autonomous Vehicles7 Oct 2023
- 2023Federated Learning for Large-Scale Scene Modeling with Neural Radiance Fields12 Sep 2023
- 2023UFed-GAN: A Secure Federated Learning Framework with Constrained Computation and Unlabeled Data10 Aug 2023
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challenges20 Jul 2023
- 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learning14 Jul 2023
- 2023FedMultimodal: A Benchmark For Multimodal Federated Learning15 Jun 2023
- 2023Masked Autoencoders are Efficient Continual Federated Learners6 Jun 2023
- 2023Federated Generalized Category Discovery23 May 2023
- 2023Survey of Federated Learning Models for Spatial-Temporal Mobility Applications9 May 2023
50 newest of 92 All 92 (JSON)
- 2021Model-Contrastive Federated Learningcited by 762
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
- 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
- 2022Divergence-aware Federated Self-Supervised Learningcited by 71
- 2021Collaborative Unsupervised Visual Representation Learning from Decentralized Datacited by 68
- 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
- 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
- 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
- 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
- 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillationcited by 40
- 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
12 most cited of 90 All 90 (JSON)
- 2026Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data18 Sep 2026
- 2026Federated Attention Autoencoders with a Stochastic Aggregation Scheme for Anomaly Detection9 Aug 2026
- 2026Sheaf-Based Federated Representation Learning8 Aug 2026
- 2026StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting26 May 2026
- 2026Provably Communication-Efficient and Privacy-Preserving Federated Graph Neural Networks25 May 2026
- 2026Semantic-based Distributed Learning for Diverse and Discriminative Representations20 Apr 2026
- 2026FedCova: Robust Federated Covariance Learning Against Noisy Labels4 Mar 2026
- 2025Local K-Similarity Constraint for Federated Learning with Label Noise9 Nov 2025
- 2025FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements14 Oct 2025
- 2025Cooperative Pseudo Labeling for Unsupervised Federated Classification11 Oct 2025
- 2025IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learning11 Aug 2025
- 2025Soft Separation and Distillation: Toward Global Uniformity in Federated Unsupervised Learning2 Aug 2025
12 newest of 90 All 90 (JSON)