mlsanityData to 7 Oct 2026

Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT

arXiv 2303.02668 ↗First posted 5 Mar 2023cs.LG cs.AI cs.DCcited by 6

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) enables multiple clients to train models collaboratively without sharing local data, which has achieved promising results in different areas, including the Internet of Things (IoT). However, end IoT devices do not have abilities to automatically annotate their collected data, which leads to the label shortage issue at the client side. To collaboratively train an FL model, we can only use a small number of labeled data stored on the server. This is a new yet practical scenario in federated learning, i.e., labels-at-server semi-supervised federated learning (SemiFL). Although several SemiFL approaches have been proposed recently, none of them can focus on the personalization issue in their model design. IoT environments make SemiFL more challenging, as we need to take device computational constraints and communication cost into consideration simultaneously. To tackle these new challenges together, we propose a novel SemiFL framework named pFedKnow. pFedKnow generates lightweight personalized client models via neural network pruning techniques to reduce communication cost. Moreover, it incorporates pretrained large models as prior knowledge to guide the aggregation of personalized client models and further enhance the framework performance. Experiment results on both image and text datasets show that the proposed pFedKnow outperforms state-of-the-art baselines as well as reducing considerable communication cost. The source code of the proposed pFedKnow is available at https://github.com/JackqqWang/pfedknow/tree/master.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 5 Mar 2023.

Builds on

31 of 41 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  9. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  10. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  11. 2017Federated Multi-Task Learningcited by 669
  12. 2017Deep Mutual Learningcited by 621
19 more
  1. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  2. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  5. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 189
  6. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 178
  7. 2020Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learningcited by 112
  8. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  9. 2019Federated Learning with Cooperating Devices: A Consensus Approach for Massive IoT Networkscited by 79
  10. 2016SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Drivingcited by 55
  11. 2021Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigmcited by 40
  12. 2018Federated Learning for Ultra-Reliable Low-Latency V2V Communicationscited by 30
  13. 2022Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoTcited by 25
  14. 2020A Federated Learning Framework for Healthcare IoT devicescited by 22
  15. 2021FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learningcited by 7
  16. 2022Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Trainingcited by 7
  17. 2017Bayesian Sparsification of Recurrent Neural Networkscited by 4
  18. 2021FedSEAL: Semi-Supervised Federated Learning with Self-Ensemble Learning and Negative Learningcited by 4
  19. 2021Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Designcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2023Towards Personalized Federated Learning via Heterogeneous Model Reassemblycited by 31
  2. 2024Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modalitycited by 14
  3. 2024Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagationcited by 5
  4. 2023SemiSFL: Split Federated Learning on Unlabeled and Non-IID Datacited by 0
  5. 2025Federated Koopman-Reservoir Learning for Large-Scale Multivariate Time-Series Anomaly Detectioncited by 0
  6. 2026A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilizationcited by 0