mlsanityData to 7 Oct 2026

Federated Learning with Matched Averaging

arXiv 2002.06440 ↗First posted 15 Feb 2020cs.LG stat.MLcited by 564

Not on the map. Why?

Abstract (arXiv)

Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, but also reduces the overall communication burden.

Key facts. Cited by 564 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 97; 2022: 112; 2023: 125; 2024: 123; 2025: 76; 2026: 31.

564 papers.

Sort:

By area:

  1. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  2. 2021Model-Contrastive Federated Learningcited by 762
  3. 2021Merging Models with Fisher-Weighted Averagingcited by 613
  4. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  5. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  6. 2021Towards Personalized Federated Learningcited by 462
  7. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  8. 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 384
  9. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  10. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  11. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  12. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
38 more
  1. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 248
  2. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  3. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  4. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  5. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  6. 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 178
  7. 2022NOTE: Robust Continual Test-time Adaptation Against Temporal Correlationcited by 150
  8. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
  9. 2021Federated Learning for Open Bankingcited by 138
  10. 2022Federated Class-Incremental Learningcited by 126
  11. 2022FedALA: Adaptive Local Aggregation for Personalized Federated Learningcited by 121
  12. 2022REPAIR: REnormalizing Permuted Activations for Interpolation Repaircited by 119
  13. 2021Local Learning Matters: Rethinking Data Heterogeneity in Federated Learningcited by 114
  14. 2024Twin-Merging: Dynamic Integration of Modular Expertise in Model Mergingcited by 114
  15. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  16. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  17. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  18. 2022Federated Learning with Label Distribution Skew via Logits Calibrationcited by 108
  19. 2023Equivariant Architectures for Learning in Deep Weight Spacescited by 106
  20. 2022Neurotoxin: Durable Backdoors in Federated Learningcited by 97
  21. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  22. 2021PPFL: Privacy-preserving Federated Learning with Trusted Execution Environmentscited by 93
  23. 2023Deep Model Fusion: A Surveycited by 89
  24. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  25. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  26. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  27. 2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 78
  28. 2022Deep Model Reassemblycited by 78
  29. 2021Personalized Federated Learning with Gaussian Processescited by 76
  30. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  31. 2021On Large-Cohort Training for Federated Learningcited by 73
  32. 2022Divergence-aware Federated Self-Supervised Learningcited by 71
  33. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  34. 2024Parameter Competition Balancing for Model Mergingcited by 71
  35. 2023FedDisco: Federated Learning with Discrepancy-Aware Collaborationcited by 66
  36. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  37. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  38. 2022Re-basin via implicit Sinkhorn differentiationcited by 61

50 most cited of 564 All 564 (JSON)