mlsanityData to 14 Oct 2026

One-shot Federated Learning without Server-side Training

arXiv 2204.12493 ↗First posted 26 Apr 2022cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has recently made significant progress as a new machine learning paradigm for privacy protection. Due to the high communication cost of traditional FL, one-shot federated learning is gaining popularity as a way to reduce communication cost between clients and the server. Most of the existing one-shot FL methods are based on Knowledge Distillation; however, distillation based approach requires an extra training phase and depends on publicly available data sets or generated pseudo samples. In this work, we consider a novel and challenging cross-silo setting: performing a single round of parameter aggregation on the local models without server-side training. In this setting, we propose an effective algorithm for Model Aggregation via Exploring Common Harmonized Optima (MA-Echo), which iteratively updates the parameters of all local models to bring them close to a common low-loss area on the loss surface, without harming performance on their own data sets at the same time. Compared to the existing methods, MA-Echo can work well even in extremely non-identical data distribution settings where the support categories of each local model have no overlapped labels with those of the others. We conduct extensive experiments on two popular image classification data sets to compare the proposed method with existing methods and demonstrate the effectiveness of MA-Echo, which clearly outperforms the state-of-the-arts. The source code can be accessed in urlhttps://github.com/FudanVI/MAEcho.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

41 of 58 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  8. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  9. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  10. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  11. 2020Adaptive Federated Optimizationcited by 1,046
  12. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
29 more
  1. 2018LEAF: A Benchmark for Federated Settingscited by 792
  2. 2021Model-Contrastive Federated Learningcited by 766
  3. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  4. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  5. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  6. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  9. 2019Agnostic Federated Learningcited by 456
  10. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  11. 2019Federated Machine Learning: Concept and Applicationscited by 412
  12. 2019Fair Resource Allocation in Federated Learningcited by 394
  13. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 387
  14. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
  15. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  16. 2018Dataset Distillationcited by 230
  17. 2019Model Fusion via Optimal Transportcited by 219
  18. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  19. 2019One-Shot Federated Learningcited by 125
  20. 2020Distilled One-Shot Federated Learningcited by 109
  21. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  22. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 100
  23. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  24. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  25. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  26. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 69
  27. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  28. 2020Federated Learning Meets Multi-objective Optimizationcited by 53
  29. 2019One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Themcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 4; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2023Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Modelcited by 33
  2. 2022Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learningcited by 29
  3. 2023FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregationcited by 22
  4. 2023Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learningcited by 21
  5. 2023Subspace based Federated Unlearningcited by 17
  6. 2024FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Modelscited by 17
  7. 2024FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Modelscited by 16
  8. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  9. 2025One-shot Federated Learning Methods: A Practical Guidecited by 6
  10. 2022Cross-domain Federated Object Detectioncited by 5
  11. 2023One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Modelscited by 5
  12. 2023Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectivescited by 3
8 more
  1. 2026The Gaussian-Head OFL Family: One-Shot Federated Learning from Client Global Statisticscited by 3
  2. 2023FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregationcited by 2
  3. 2025Towards One-shot Federated Learning: Advances, Challenges, and Future Directionscited by 2
  4. 2022Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learningcited by 1
  5. 2024One-Shot Collaborative Data Distillationcited by 1
  6. 2023CRS-FL: Conditional Random Sampling for Communication-Efficient and Privacy-Preserving Federated Learningcited by 0
  7. 2025One-Shot Federated Unsupervised Domain Adaptation with Scaled Entropy Attention and Multi-Source Smoothed Pseudo Labelingcited by 0
  8. 2025Tackling Federated Unlearning as a Parameter Estimation Problemcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024One-shot Federated Learning via Synthetic Distiller-Distillate Communication
  2. 2024Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Models
  3. 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation
  4. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency
  5. 2025One-Shot Federated Learning with Classifier-Free Diffusion Models
  6. 2025Towards a Larger Model via One-Shot Federated Learning on Heterogeneous Client Models
  7. 2024Federated Learning with a Single Shared Image
  8. 2022Federated Learning with Lossy Distributed Source Coding: Analysis and Optimization ↗