mlsanityData to 7 Oct 2026

Multi-Task Distributed Learning using Vision Transformer with Random Patch Permutation

arXiv 2204.03500 ↗First posted 7 Apr 2022cs.LG cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

The widespread application of artificial intelligence in health research is currently hampered by limitations in data availability. Distributed learning methods such as federated learning (FL) and shared learning (SL) are introduced to solve this problem as well as data management and ownership issues with their different strengths and weaknesses. The recent proposal of federated split task-agnostic (FeSTA) learning tries to reconcile the distinct merits of FL and SL by enabling the multi-task collaboration between participants through Vision Transformer (ViT) architecture, but they suffer from higher communication overhead. To address this, here we present a multi-task distributed learning using ViT with random patch permutation. Instead of using a CNN based head as in FeSTA, p-FeSTA adopts a randomly permuting simple patch embedder, improving the multi-task learning performance without sacrificing privacy. Experimental results confirm that the proposed method significantly enhances the benefit of multi-task collaboration, communication efficiency, and privacy preservation, shedding light on practical multi-task distributed learning in the field of medical imaging.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

37 of 51 references.

References · 37

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2019Deep Leakage from Gradientscited by 1,004
  10. 2020Pre-Trained Image Processing Transformercited by 891
  11. 2018How To Backdoor Federated Learningcited by 745
  12. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
25 more
  1. 2020The Future of Digital Health with Federated Learningcited by 625
  2. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 449
  3. 2021Intriguing Properties of Vision Transformerscited by 446
  4. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  5. 2020iDLG: Improved Deep Leakage from Gradientscited by 367
  6. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  7. 2020SplitFed: When Federated Learning Meets Split Learningcited by 308
  8. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  9. 2020Threats to Federated Learning: A Surveycited by 171
  10. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  11. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  12. 2020Unleashing the Tiger: Inference Attacks on Split Learningcited by 104
  13. 2020Learning to Detect Malicious Clients for Robust Federated Learningcited by 86
  14. 2021Federated Learning: Opportunities and Challengescited by 83
  15. 2020BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patientscited by 81
  16. 2020End-to-End Evaluation of Federated Learning and Split Learning for Internet of Thingscited by 64
  17. 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
  18. 2020Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneitycited by 56
  19. 2020Weakly Supervised Lesion Localization With Probabilistic-CAM Poolingcited by 35
  20. 2021Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantificationcited by 21
  21. 2021Membership Inference Attacks on Deep Regression Models for Neuroimagingcited by 19
  22. 2021Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Trainingcited by 17
  23. 2022Feature Space Hijacking Attacks against Differentially Private Split Learningcited by 14
  24. 2019Convolutional Neural Networks on Randomized Datacited by 6
  25. 2019Privacy-Preserving Machine Learning Using EtC Imagescited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 5; 2025: 2.

10 papers.

  1. 2024From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcarecited by 16
  2. 2023Recent Methodological Advances in Federated Learning for Healthcarecited by 11
  3. 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
  4. 2025Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Taskscited by 2
  5. 2024Private Knowledge Sharing in Distributed Learning: A Surveycited by 1
  6. 2024Heterogeneous Federated Learning with Splited Language Modelcited by 1
  7. 2024Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Trainingcited by 1
  8. 2023Single-round Self-supervised Distributed Learning using Vision Transformercited by 0
  9. 2024Privacy-Preserving Vision Transformer Using Images Encrypted with Restricted Random Permutation Matricescited by 0
  10. 2025MedSegNet10: A Publicly Accessible Network Repository for Split Federated Medical Image Segmentationcited by 0