# Tackling Heterogeneity in Medical Federated learning via Vision Transformers

arXiv: https://arxiv.org/abs/2310.09444 · HTML: https://mlsanity.com/paper/2310.09444/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 13 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Privacy-preserving Federated Brain Tumour Segmentation](https://mlsanity.com/paper/1910.00962/) (2019)
- [Subgraph Federated Learning with Missing Neighbor Generation](https://mlsanity.com/paper/2106.13430/) (2021)
- [Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results](https://mlsanity.com/paper/2001.05647/) (2020)
- [Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2106.06047/) (2021)
- [FedSplit: An algorithmic framework for fast federated optimization](https://mlsanity.com/paper/2005.05238/) (2020)
- [Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning](https://mlsanity.com/paper/2103.02148/) (2021)
- [HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images](https://mlsanity.com/paper/2112.10775/) (2021)
- [Federated Learning for Breast Density Classification: A Real-World Implementation](https://mlsanity.com/paper/2009.01871/) (2020)
- [Fast-Convergent Federated Learning](https://mlsanity.com/paper/2007.13137/) (2020)
- [Federated Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2204.10983/) (2022)
- [SplitAVG: A heterogeneity-aware federated deep learning method for medical imaging](https://mlsanity.com/paper/2107.02375/) (2021)
- [Multi-task Federated Learning for Heterogeneous Pancreas Segmentation](https://mlsanity.com/paper/2108.08537/) (2021)
- [HHHFL: Hierarchical Heterogeneous Horizontal Federated Learning for Electroencephalography](https://mlsanity.com/paper/1909.05784/) (2019)
- [A Comparative Study of Federated Learning Models for COVID-19 Detection](https://mlsanity.com/paper/2303.16141/) (2023)
- [Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data](https://mlsanity.com/paper/2207.03448/) (2022)
- [Deep Learning Approach for Early Stage Lung Cancer Detection](https://mlsanity.com/paper/2302.02456/) (2023)

## Abstract (arXiv)

Optimization-based regularization methods have been effective in addressing the challenges posed by data heterogeneity in medical federated learning, particularly in improving the performance of underrepresented clients. However, these methods often lead to lower overall model accuracy and slower convergence rates. In this paper, we demonstrate that using Vision Transformers can substantially improve the performance of underrepresented clients without a significant trade-off in overall accuracy. This improvement is attributed to the Vision transformer's ability to capture long-range dependencies within the input data.