mlsanityData to 14 Oct 2026

dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis

arXiv 2503.10412 ↗First posted 13 Mar 2025cs.LG cs.AIcited by 9

On the map: Mixture-of-experts → Federated mixture-of-experts

Abstract (arXiv)

Federated learning has wide applications in the medical field. It enables knowledge sharing among different healthcare institutes while protecting patients' privacy. However, existing federated learning systems are typically centralized, requiring clients to upload client-specific knowledge to a central server for aggregation. This centralized approach would integrate the knowledge from each client into a centralized server, and the knowledge would be already undermined during the centralized integration before it reaches back to each client. Besides, the centralized approach also creates a dependency on the central server, which may affect training stability if the server malfunctions or connections are unstable. To address these issues, we propose a decentralized federated learning framework named dFLMoE. In our framework, clients directly exchange lightweight head models with each other. After exchanging, each client treats both local and received head models as individual experts, and utilizes a client-specific Mixture of Experts (MoE) approach to make collective decisions. This design not only reduces the knowledge damage with client-specific aggregations but also removes the dependency on the central server to enhance the robustness of the framework. We validate our framework on multiple medical tasks, demonstrating that our method evidently outperforms state-of-the-art approaches under both model homogeneity and heterogeneity settings.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 58 of its 81 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

58 of 81 references.

References · 58

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2017Squeeze-and-Excitation Networkscited by 4,789
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  10. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  11. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
38 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  3. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  4. 2019UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentationcited by 586
  5. 2019Federated Learning with Personalization Layerscited by 541
  6. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  9. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  10. 2020Adaptive Personalized Federated Learningcited by 358
  11. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  12. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  13. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  14. 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 261
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  16. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  17. 2021Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformerscited by 159
  18. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  19. 2019BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learningcited by 117
  20. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 109
  21. 2021Personalized Federated Learning through Local Memorizationcited by 57
  22. 2020Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computingcited by 52
  23. 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
  24. 2022Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentationcited by 42
  25. 2020FedCoin: A Peer-to-Peer Payment System for Federated Learningcited by 39
  26. 2022Personalizing Federated Medical Image Segmentation via Local Calibrationcited by 29
  27. 2021PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videoscited by 25
  28. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  29. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  30. 2024MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysiscited by 14
  31. 2019Privacy-Preserving Blockchain-Based Federated Learning for IoT Devicescited by 12
  32. 2022BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Frameworkcited by 11
  33. 2024AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-maskingcited by 10
  34. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  35. 2024Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluationcited by 9
  36. 2023SHISRCNet: Super-resolution And Classification Network For Low-resolution Breast Cancer Histopathology Imagecited by 7
  37. 2022Discriminative Kernel Convolution Network for Multi-Label Ophthalmic Disease Detection on Imbalanced Fundus Image Datasetcited by 6
  38. 2022Federated Learning with Privacy-Preserving Ensemble Attention Distillationcited by 6

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneitycited by 4
  2. 2026The Rise of Sparse Mixture-of-Experts: A Survey from Algorithmic Foundations to Decentralized Architectures and Vertical Domain Applicationscited by 2
  3. 2026Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRAcited by 2
  4. 2025Robust Client-Server Watermarking for Split Federated Learningcited by 0
  5. 2026AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learningcited by 0
  6. 2026Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Expertscited by 0
  7. 2026FoMoE: Breaking the Full-Replica Barrier with a Federation of MoEscited by 0
  8. 2026Fisher-Routed Mixture of Experts for Federated Class-Incremental Learningcited by 0
  9. 2026Regime-Aware Router Learning for Multi-Domain Medical Image Generation with Diffusion Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification
  2. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
  3. 2025FedAgentBench: Towards Automating Real-world Federated Medical Image Analysis with Server-Client LLM Agents
  4. 2023Federated Learning for Data and Model Heterogeneity in Medical Imaging
  5. 2026FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems
  6. 2020Specialized federated learning using a mixture of experts
  7. 2026FedBiCross: Personalized One-Shot Federated Learning on Medical Images
  8. 2022MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare