# Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT

arXiv: https://arxiv.org/abs/2303.02668 · HTML: https://mlsanity.com/paper/2303.02668/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Q8BERT: Quantized 8Bit BERT](https://mlsanity.com/paper/1910.06188/) (2019)
- [Compressing Deep Convolutional Networks using Vector Quantization](https://mlsanity.com/paper/1412.6115/) (2014)
- [FetchSGD: Communication-Efficient Federated Learning with Sketching](https://mlsanity.com/paper/2007.07682/) (2020)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks](https://mlsanity.com/paper/2104.08815/) (2021)
- [Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networks](https://mlsanity.com/paper/1912.13163/) (2019)
- [SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving](https://mlsanity.com/paper/1612.01051/) (2016)
- [Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm](https://mlsanity.com/paper/2104.08682/) (2021)
- [Federated Learning for Ultra-Reliable Low-Latency V2V Communications](https://mlsanity.com/paper/1805.09253/) (2018)
- [Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT](https://mlsanity.com/paper/2202.01512/) (2022)
- [A Federated Learning Framework for Healthcare IoT devices](https://mlsanity.com/paper/2005.05083/) (2020)
- [FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning](https://mlsanity.com/paper/2109.05612/) (2021)
- [Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training](https://mlsanity.com/paper/2201.12976/) (2022)
- [Bayesian Sparsification of Recurrent Neural Networks](https://mlsanity.com/paper/1708.00077/) (2017)
- [FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learning](https://mlsanity.com/paper/2110.07829/) (2021)

## Built on by (most cited first, 6 of 6)

- [Towards Personalized Federated Learning via Heterogeneous Model Reassembly](https://mlsanity.com/paper/2308.08643/) (2023)
- [Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality](https://mlsanity.com/paper/2406.11048/) (2024)
- [Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation](https://mlsanity.com/paper/2401.15874/) (2024)
- [SemiSFL: Split Federated Learning on Unlabeled and Non-IID Data](https://mlsanity.com/paper/2307.15870/) (2023)
- [Federated Koopman-Reservoir Learning for Large-Scale Multivariate Time-Series Anomaly Detection](https://mlsanity.com/paper/2503.11255/) (2025)
- [A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization](https://mlsanity.com/paper/2609.25471/) (2026)

## Abstract (arXiv)

Federated learning (FL) enables multiple clients to train models collaboratively without sharing local data, which has achieved promising results in different areas, including the Internet of Things (IoT). However, end IoT devices do not have abilities to automatically annotate their collected data, which leads to the label shortage issue at the client side. To collaboratively train an FL model, we can only use a small number of labeled data stored on the server. This is a new yet practical scenario in federated learning, i.e., labels-at-server semi-supervised federated learning (SemiFL). Although several SemiFL approaches have been proposed recently, none of them can focus on the personalization issue in their model design. IoT environments make SemiFL more challenging, as we need to take device computational constraints and communication cost into consideration simultaneously. To tackle these new challenges together, we propose a novel SemiFL framework named pFedKnow. pFedKnow generates lightweight personalized client models via neural network pruning techniques to reduce communication cost. Moreover, it incorporates pretrained large models as prior knowledge to guide the aggregation of personalized client models and further enhance the framework performance. Experiment results on both image and text datasets show that the proposed pFedKnow outperforms state-of-the-art baselines as well as reducing considerable communication cost. The source code of the proposed pFedKnow is available at https://github.com/JackqqWang/pfedknow/tree/master.