mlsanityData to 14 Oct 2026

SelfFed: Self-Supervised Federated Learning for Data Heterogeneity and Label Scarcity in Medical Images

arXiv 2307.01514 ↗First posted 4 Jul 2023cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning in the federated learning paradigm has been gaining a lot of interest both in industry and research due to the collaborative learning capability on unlabeled yet isolated data. However, self-supervised based federated learning strategies suffer from performance degradation due to label scarcity and diverse data distributions, i.e., data heterogeneity. In this paper, we propose the SelfFed framework for medical images to overcome data heterogeneity and label scarcity issues. The first phase of the SelfFed framework helps to overcome the data heterogeneity issue by leveraging the pre-training paradigm that performs augmentative modeling using Swin Transformer-based encoder in a decentralized manner. The label scarcity issue is addressed by fine-tuning paradigm that introduces a contrastive network and a novel aggregation strategy. We perform our experimental analysis on publicly available medical imaging datasets to show that SelfFed performs better when compared to existing baselines and works. Our method achieves a maximum improvement of 8.8% and 4.1% on Retina and COVID-FL datasets on non-IID datasets. Further, our proposed method outperforms existing baselines even when trained on a few (10%) labeled instances.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

27 of 38 references.

References · 27

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
15 more
  1. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  2. 2020Adaptive Federated Optimizationcited by 1,046
  3. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  5. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  6. 2020Personalized Cross-Silo Federated Learning on Non-IID Datacited by 283
  7. 2021Contrastive Learning based Hybrid Networks for Long-Tailed Image Classificationcited by 130
  8. 2020Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Datacited by 125
  9. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
  10. 2023Federated Learning for Medical Image Analysis: A Surveycited by 78
  11. 2022On the Importance and Applicability of Pre-Training for Federated Learningcited by 71
  12. 2022Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imagingcited by 39
  13. 2022Towards Energy Efficient Distributed Federated Learning for 6G Networkscited by 7
  14. 2023Self-Supervised Federated Learning for Fast MR Imagingcited by 5
  15. 2022Pseudo-Data based Self-Supervised Federated Learning for Classification of Histopathological Imagescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 3.

4 papers.

  1. 2024Robust and Explainable Framework to Address Data Scarcity in Diagnostic Imagingcited by 1
  2. 2026Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classificationcited by 0
  3. 2026FedDP-PALD: A Privacy-Preserving Federated Latent Diffusion Framework with Prototype Aggregation for Medical Data Synthesiscited by 0
  4. 2026GCA: Global Centroid Alignment in Federated Learningcited by 0