mlsanityData to 7 Oct 2026

Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Design

arXiv 2110.13388 ↗First posted 26 Oct 2021cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) allows edge devices (or clients) to keep data locally while simultaneously training a shared high-quality global model. However, current research is generally based on an assumption that the training data of local clients have ground-truth. Furthermore, FL faces the challenge of statistical heterogeneity, i.e., the distribution of the client's local training data is non-independent identically distributed (non-IID). In this paper, we present a robust semi-supervised FL system design, where the system aims to solve the problem of data availability and non-IID in FL. In particular, this paper focuses on studying the labels-at-server scenario where there is only a limited amount of labeled data on the server and only unlabeled data on the clients. In our system design, we propose a novel method to tackle the problems, which we refer to as Federated Mixing (FedMix). FedMix improves the naive combination of FL and semi-supervised learning methods and designs parameter decomposition strategies for disjointed learning of labeled, unlabeled data, and global models. To alleviate the non-IID problem, we propose a novel aggregation rule based on the frequency of the client's participation in training, namely the FedFreq aggregation algorithm, which can adjust the weight of the corresponding local model according to this frequency. Extensive evaluations conducted on CIFAR-10 dataset show that the performance of our proposed method is significantly better than those of the current baseline. It is worth noting that our system is robust to different non-IID levels of client data.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 26 Oct 2021.

Builds on

28 of 41 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  4. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2018Federated Learning with Non-IID Datacited by 1,099
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  10. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  11. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  12. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
16 more
  1. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  2. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  5. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  6. 2021A Survey on Deep Semi-supervised Learningcited by 250
  7. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  8. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  9. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  10. 2019Reliable Federated Learning for Mobile Networkscited by 92
  11. 2020Privacy-preserving Traffic Flow Prediction: A Federated Learning Approachcited by 82
  12. 2020GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphscited by 64
  13. 2020Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approachcited by 56
  14. 2017Adversarial Dropout for Supervised and Semi-supervised Learningcited by 38
  15. 2020Federated Learning for 6G Communications: Challenges, Methods, and Future Directionscited by 32
  16. 2020Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospectivecited by 27

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1.

3 papers.

  1. 2023Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoTcited by 6
  2. 2022FedNST: Federated Noisy Student Training for Automatic Speech Recognitioncited by 5
  3. 2025Prediction-space knowledge markets for communication-efficient federated learning on multimedia taskscited by 0