FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional FL introduces significant computation and communication overhead, which is unfriendly for resource-constrained edge devices. One-shot FL has emerged as a promising approach to mitigate communication overhead, and model-heterogeneous FL solves the problem of diverse computing resources across clients. However, existing methods face challenges in effectively managing model-heterogeneous one-shot FL, often leading to unsatisfactory global model performance or reliance on auxiliary datasets. To address these challenges, we propose a novel FL framework named FedMHO, which leverages deep classification models on resource-sufficient clients and lightweight generative models on resource-constrained devices. On the server side, FedMHO involves a two-stage process that includes data generation and knowledge fusion. Furthermore, we introduce FedMHO-MD and FedMHO-SD to mitigate the knowledge-forgetting problem during the knowledge fusion stage, and an unsupervised data optimization solution to improve the quality of synthetic samples. Comprehensive experiments demonstrate the effectiveness of our methods, as they outperform state-of-the-art baselines in various experimental setups.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 12 Feb 2025.
Builds on
47 of 69 references.
References · 47
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2019Advances and Open Problems in Federated Learningcited by 3,046
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
- 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 848
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2017Deep Mutual Learningcited by 628
35 more
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
- 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
- 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
- 2021See through Gradients: Image Batch Recovery via GradInversioncited by 326
- 2021A Field Guide to Federated Optimizationcited by 315
- 2021Distilling Knowledge via Knowledge Reviewcited by 288
- 2018Distributed learning of deep neural network over multiple agentscited by 279
- 2018Dataset Distillationcited by 230
- 2019Data-Free Learning of Student Networkscited by 204
- 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
- 2021Does Knowledge Distillation Really Work?cited by 190
- 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 166
- 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
- 2019One-Shot Federated Learningcited by 125
- 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
- 2021Learning Placeholders for Open-Set Recognitioncited by 121
- 2020Distilled One-Shot Federated Learningcited by 109
- 2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
- 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
- 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
- 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
- 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 66
- 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 35
- 2022Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillationcited by 31
- 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
- 2023DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learningcited by 24
- 2019One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Themcited by 23
- 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
- 2022Exploring the Distributed Knowledge Congruence in Proxy-data-free Federated Distillationcited by 22
- 2019Man-in-the-Middle Attacks against Machine Learning Classifiers via Malicious Generative Modelscited by 19
- 2024One-shot Federated Learning via Synthetic Distiller-Distillate Communicationcited by 12
- 2023FLrce: Resource-Efficient Federated Learning with Early-Stopping Strategycited by 10
- 2024Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clientscited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.