# SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling

arXiv: https://arxiv.org/abs/2108.09412 · HTML: https://mlsanity.com/paper/2108.09412/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [FedML: A Research Library and Benchmark for Federated Machine Learning](https://mlsanity.com/paper/2007.13518/) (2020)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [One-Shot Federated Learning](https://mlsanity.com/paper/1902.11175/) (2019)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [A generic framework for privacy preserving deep learning](https://mlsanity.com/paper/1811.04017/) (2018)

## Built on by (most cited first, 29 of 29)

- [When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions](https://mlsanity.com/paper/2306.15546/) (2023)
- [RSCFed: Random Sampling Consensus Federated Semi-supervised Learning](https://mlsanity.com/paper/2203.13993/) (2022)
- [Federated Generalized Category Discovery](https://mlsanity.com/paper/2305.14107/) (2023)
- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling](https://mlsanity.com/paper/2203.08810/) (2022)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators](https://mlsanity.com/paper/2307.05358/) (2023)
- [Federated Semi-Supervised Learning with Prototypical Networks](https://mlsanity.com/paper/2205.13921/) (2022)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [Towards Unbiased Training in Federated Open-world Semi-supervised Learning](https://mlsanity.com/paper/2305.00771/) (2023)
- [Robust Training of Federated Models with Extremely Label Deficiency](https://mlsanity.com/paper/2402.14430/) (2024)
- [A Mutual Information Perspective on Federated Contrastive Learning](https://mlsanity.com/paper/2405.02081/) (2024)
- [Learning Dynamic Graphs from All Contextual Information for Accurate Point-of-Interest Visit Forecasting](https://mlsanity.com/paper/2306.15927/) (2023)
- [Little is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2310.05696/) (2023)
- [Estimating before Debiasing: A Bayesian Approach to Detaching Prior Bias in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2405.19789/) (2024)
- [Rethinking Semi-Supervised Federated Learning: How to co-train fully-labeled and fully-unlabeled client imaging data](https://mlsanity.com/paper/2310.18815/) (2023)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [Dual Class-Aware Contrastive Federated Semi-Supervised Learning](https://mlsanity.com/paper/2211.08914/) (2022)
- [Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice](https://mlsanity.com/paper/2510.12595/) (2025)
- [Fair Federated Data Clustering through Personalization: Bridging the Gap between Diverse Data Distributions](https://mlsanity.com/paper/2407.04302/) (2024)
- [Unlock the Power of Unlabeled Data in Language Driving Model](https://mlsanity.com/paper/2503.10586/) (2025)
- [Uncertainty Minimization for Personalized Federated Semi-Supervised Learning](https://mlsanity.com/paper/2205.02438/) (2022)
- [Federated Semi-Supervised Learning with Annotation Heterogeneity](https://mlsanity.com/paper/2303.02445/) (2023)
- [SemiSFL: Split Federated Learning on Unlabeled and Non-IID Data](https://mlsanity.com/paper/2307.15870/) (2023)
- [Learning Unlabeled Clients Divergence for Federated Semi-Supervised Learning via Anchor Model Aggregation](https://mlsanity.com/paper/2407.10327/) (2024)
- [Diffusion Model-Based Data Synthesis Aided Federated Semi-Supervised Learning](https://mlsanity.com/paper/2501.02219/) (2025)
- [Unified Alignment Protocol: Making Sense of the Unlabeled Data in New Domains](https://mlsanity.com/paper/2505.21010/) (2025)
- [Closer to Reality: Practical Semi-Supervised Federated Learning for Foundation Model Adaptation](https://mlsanity.com/paper/2508.16568/) (2025)
- [Closing the Alignment-Maturity Gap in Federated Prototype Learning](https://mlsanity.com/paper/2606.02172/) (2026)

## Abstract (arXiv)

Federated learning enables multiple clients, such as mobile phones and organizations, to collaboratively learn a shared model for prediction while protecting local data privacy. However, most recent research and applications of federated learning assume that all clients have fully labeled data, which is impractical in real-world settings. In this work, we focus on a new scenario for cross-silo federated learning, where data samples of each client are partially labeled. We borrow ideas from semi-supervised learning methods where a large amount of unlabeled data is utilized to improve the model's accuracy despite limited access to labeled examples. We propose a new framework dubbed SemiFed that unifies two dominant approaches for semi-supervised learning: consistency regularization and pseudo-labeling. SemiFed first applies advanced data augmentation techniques to enforce consistency regularization and then generates pseudo-labels using the model's predictions during training. SemiFed takes advantage of the federation so that for a given image, the pseudo-label holds only if multiple models from different clients produce a high-confidence prediction and agree on the same label. Extensive experiments on two image benchmarks demonstrate the effectiveness of our approach under both homogeneous and heterogeneous data distribution settings