mlsanityData to 7 Oct 2026

Tackling Heterogeneity in Medical Federated learning via Vision Transformers

arXiv 2310.09444 ↗First posted 13 Oct 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Optimization-based regularization methods have been effective in addressing the challenges posed by data heterogeneity in medical federated learning, particularly in improving the performance of underrepresented clients. However, these methods often lead to lower overall model accuracy and slower convergence rates. In this paper, we demonstrate that using Vision Transformers can substantially improve the performance of underrepresented clients without a significant trade-off in overall accuracy. This improvement is attributed to the Vision transformer's ability to capture long-range dependencies within the input data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 13 Oct 2023.

Builds on

30 of 38 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
  8. 2021Model-Contrastive Federated Learningcited by 762
  9. 2020The Future of Digital Health with Federated Learningcited by 625
  10. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  11. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  12. 2019Federated Learning with Personalization Layerscited by 533
18 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  2. 2022Transformers in Medical Imaging: A Surveycited by 245
  3. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  4. 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
  5. 2020Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Resultscited by 122
  6. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  7. 2020FedSplit: An algorithmic framework for fast federated optimizationcited by 108
  8. 2021Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learningcited by 78
  9. 2020Federated Learning for Breast Density Classification: A Real-World Implementationcited by 62
  10. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 62
  11. 2020Fast-Convergent Federated Learningcited by 61
  12. 2022Federated Contrastive Learning for Volumetric Medical Image Segmentationcited by 32
  13. 2021SplitAVG: A heterogeneity-aware federated deep learning method for medical imagingcited by 19
  14. 2021Multi-task Federated Learning for Heterogeneous Pancreas Segmentationcited by 19
  15. 2019HHHFL: Hierarchical Heterogeneous Horizontal Federated Learning for Electroencephalographycited by 15
  16. 2023A Comparative Study of Federated Learning Models for COVID-19 Detectioncited by 3
  17. 2022Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Datacited by 2
  18. 2023Deep Learning Approach for Early Stage Lung Cancer Detectioncited by 2

Built on by

Not cited yet.