mlsanityData to 7 Oct 2026

MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis

arXiv 2405.06822 ↗First posted 10 May 2024cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

Federated learning is widely used in medical applications for training global models without needing local data access. However, varying computational capabilities and network architectures (system heterogeneity), across clients pose significant challenges in effectively aggregating information from non-independently and identically distributed (non-IID) data. Current federated learning methods using knowledge distillation require public datasets, raising privacy and data collection issues. Additionally, these datasets require additional local computing and storage resources, which is a burden for medical institutions with limited hardware conditions. In this paper, we introduce a novel federated learning paradigm, named Model Heterogeneous personalized Federated Learning via Injection and Distillation (MH-pFLID). Our framework leverages a lightweight messenger model that carries concentrated information to collect the information from each client. We also develop a set of receiver and transmitter modules to receive and send information from the messenger model, so that the information could be injected and distilled with efficiency.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 46 of its 60 references that are in the map. First posted on arXiv 10 May 2024.

Builds on

46 of 60 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  11. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
34 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  2. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  3. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  4. 2019UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentationcited by 583
  5. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  6. 2019Federated Learning with Personalization Layerscited by 533
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  9. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
  10. 2020Adaptive Personalized Federated Learningcited by 353
  11. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  12. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  13. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 282
  14. 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 260
  15. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  16. 2019FedHealth: A Federated Transfer Learning Framework for Wearable Healthcarecited by 182
  17. 2021Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformerscited by 159
  18. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  19. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  20. 2021Personalized Federated Learning through Local Memorizationcited by 57
  21. 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 55
  22. 2020Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computingcited by 52
  23. 2022Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentationcited by 41
  24. 2022Personalizing Federated Medical Image Segmentation via Local Calibrationcited by 28
  25. 2021PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videoscited by 25
  26. 2023Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learningcited by 23
  27. 2023High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decompositioncited by 18
  28. 2022TC-SKNet with GridMask for Low-complexity Classification of Acoustic scenecited by 12
  29. 2020Deformable Gabor Feature Networks for Biomedical Image Classificationcited by 10
  30. 2022Progressive Multi-view Human Mesh Recovery with Self-Supervisioncited by 9
  31. 2023pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharingcited by 9
  32. 2024Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluationcited by 9
  33. 2024FSC: Few-point Shape Completioncited by 6
  34. 2024Federated Data Modelcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 3; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcarecited by 16
  2. 2024Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Modelscited by 16
  3. 2024Frontiers in Intelligent Colonoscopycited by 13
  4. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  5. 2025dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysiscited by 8
  6. 2025Federated Learning for Medical Image Classification: A Comprehensive Benchmarkcited by 6
  7. 2024pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentationcited by 5
  8. 2024MH-pFLGB: Model Heterogeneous personalized Federated Learning via Global Bypass for Medical Image Analysiscited by 4
  9. 2024Divide and Fuse: Body Part Mesh Recovery from Partially Visible Human Imagescited by 1
  10. 2026FedRio: Personalized Federated Social Bot Detection via Cooperative Reinforced Contrastive Adversarial Distillationcited by 1
  11. 2026Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneitycited by 1
  12. 2024FedDW: Distilling Weights through Consistency Optimization in Heterogeneous Federated Learningcited by 0
2 more
  1. 2025Prompt Mechanisms in Medical Imaging: A Comprehensive Surveycited by 0
  2. 2026FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learningcited by 0