mlsanityData to 7 Oct 2026

SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision

arXiv 2110.02470 ↗First posted 6 Oct 2021cs.LG cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is transforming the ML training ecosystem from a centralized over-the-cloud setting to distributed training over edge devices in order to strengthen data privacy. An essential but rarely studied challenge in FL is label deficiency at the edge. This problem is even more pronounced in FL compared to centralized training due to the fact that FL users are often reluctant to label their private data. Furthermore, due to the heterogeneous nature of the data at edge devices, it is crucial to develop personalized models. In this paper we propose self-supervised federated learning (SSFL), a unified self-supervised and personalized federated learning framework, and a series of algorithms under this framework which work towards addressing these challenges. First, under the SSFL framework, we demonstrate that the standard FedAvg algorithm is compatible with recent breakthroughs in centralized self-supervised learning such as SimSiam networks. Moreover, to deal with data heterogeneity at the edge devices in this framework, we have innovated a series of algorithms that broaden existing supervised personalization algorithms into the setting of self-supervised learning. We further propose a novel personalized federated self-supervised learning algorithm, Per-SSFL, which balances personalization and consensus by carefully regulating the distance between the local and global representations of data. To provide a comprehensive comparative analysis of all proposed algorithms, we also develop a distributed training system and related evaluation protocol for SSFL. Our findings show that the gap of evaluation accuracy between supervised learning and unsupervised learning in FL is both small and reasonable. The performance comparison indicates the representation regularization-based personalization method is able to outperform other variants.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

31 of 41 references.

References · 31

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  12. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
19 more
  1. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  2. 2020Adaptive Federated Optimizationcited by 1,036
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  5. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  8. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  9. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  10. 2021A Field Guide to Federated Optimizationcited by 312
  11. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  12. 2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 298
  13. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  14. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  15. 2020Federated Unsupervised Representation Learningcited by 92
  16. 2021SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classificationcited by 68
  17. 2020Federated Self-Supervised Learning of Multi-Sensor Representations for Embedded Intelligencecited by 39
  18. 2020Semi-supervised Federated Learning for Activity Recognitioncited by 18
  19. 2021Self-supervised Cross-silo Federated Neural Architecture Searchcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 9; 2024: 2; 2025: 1; 2026: 1.

26 papers.

Sort:

  1. 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
  2. 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
  3. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  4. 2022A Hybrid Self-Supervised Learning Framework for Vertical Federated Learningcited by 17
  5. 2022Federated Self-supervised Learning for Heterogeneous Clientscited by 16
  6. 2022Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?cited by 12
  7. 2023Federated Learning without Full Labels: A Surveycited by 12
  8. 2023Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulatorscited by 9
  9. 2022Federated Semi-Supervised Learning with Prototypical Networkscited by 8
  10. 2023NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clientscited by 7
  11. 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
  12. 2023Principled Federated Domain Adaptation: Gradient Projection and Auto-Weightingcited by 6
14 more
  1. 2023Towards Unbiased Training in Federated Open-world Semi-supervised Learningcited by 6
  2. 2022Bidirectional Contrastive Split Learning for Visual Question Answeringcited by 4
  3. 2023Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labelscited by 4
  4. 2023Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging datacited by 4
  5. 2023Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneitycited by 4
  6. 2022Dual Class-Aware Contrastive Federated Semi-Supervised Learningcited by 3
  7. 2022Federated Training of Dual Encoding Models on Small Non-IID Client Datasetscited by 2
  8. 2024Fair Federated Data Clustering through Personalization: Bridging the Gap between Diverse Data Distributionscited by 1
  9. 2025IPBA: Imperceptible Perturbation Backdoor Attack in Federated Self-Supervised Learningcited by 1
  10. 2022Federated Graph Representation Learning using Self-Supervisioncited by 0
  11. 2022Federated Unsupervised Visual Representation Learning via Exploiting General Content and Personal Stylecited by 0
  12. 2023Adversarial Predictions of Data Distributions Across Federated Internet-of-Things Devicescited by 0
  13. 2024Federated Learning with New Knowledge: Fundamentals, Advances, and Futurescited by 0
  14. 2026ProxyFL: A Proxy-Guided Framework for Federated Semi-Supervised Learningcited by 0