mlsanityData to 7 Oct 2026

Moss: Proxy Model-based Full-Weight Aggregation in Federated Learning with Heterogeneous Models

arXiv 2503.10218 ↗First posted 13 Mar 2025cs.LG cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern Federated Learning (FL) has become increasingly essential for handling highly heterogeneous mobile devices. Current approaches adopt a partial model aggregation paradigm that leads to sub-optimal model accuracy and higher training overhead. In this paper, we challenge the prevailing notion of partial-model aggregation and propose a novel "full-weight aggregation" method named Moss, which aggregates all weights within heterogeneous models to preserve comprehensive knowledge. Evaluation across various applications demonstrates that Moss significantly accelerates training, reduces on-device training time and energy consumption, enhances accuracy, and minimizes network bandwidth utilization when compared to state-of-the-art baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 56 of its 88 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

56 of 88 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  7. 2020Knowledge Distillation: A Surveycited by 1,578
  8. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  9. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  10. 2018Federated Learning with Non-IID Datacited by 1,099
  11. 2020Adaptive Federated Optimizationcited by 1,036
  12. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
38 more
  1. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  3. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  4. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  5. 2018Rethinking the Value of Network Pruningcited by 595
  6. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  7. 2019Similarity-Preserving Knowledge Distillationcited by 507
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  9. 2019On the Efficacy of Knowledge Distillationcited by 377
  10. 2019A Comprehensive Overhaul of Feature Distillationcited by 341
  11. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  12. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 330
  13. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  14. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  15. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  16. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  17. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  18. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 184
  19. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 162
  20. 2021FedScale: Benchmarking Model and System Performance of Federated Learning at Scalecited by 135
  21. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  22. 2019Continual Learning via Neural Pruningcited by 101
  23. 2018Federated Learning for Keyword Spottingcited by 95
  24. 2021PPFL: Privacy-preserving Federated Learning with Trusted Execution Environmentscited by 93
  25. 2019Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfercited by 91
  26. 2020Learning to Detect Malicious Clients for Robust Federated Learningcited by 86
  27. 2022Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learningcited by 81
  28. 2021FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusioncited by 49
  29. 2019FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edgecited by 48
  30. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  31. 2019Learning What and Where to Transfercited by 43
  32. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  33. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 36
  34. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  35. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  36. 2022Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloadingcited by 29
  37. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  38. 2021Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errorscited by 23

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.