mlsanityData to 7 Oct 2026

Personalizing Federated Instrument Segmentation with Visual Trait Priors in Robotic Surgery

arXiv 2408.03208 ↗First posted 6 Aug 2024cs.CV cs.AI cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Personalized federated learning (PFL) for surgical instrument segmentation (SIS) is a promising approach. It enables multiple clinical sites to collaboratively train a series of models in privacy, with each model tailored to the individual distribution of each site. Existing PFL methods rarely consider the personalization of multi-headed self-attention, and do not account for appearance diversity and instrument shape similarity, both inherent in surgical scenes. We thus propose PFedSIS, a novel PFL method with visual trait priors for SIS, incorporating global-personalized disentanglement (GPD), appearance-regulation personalized enhancement (APE), and shape-similarity global enhancement (SGE), to boost SIS performance in each site. GPD represents the first attempt at head-wise assignment for multi-headed self-attention personalization. To preserve the unique appearance representation of each site and gradually leverage the inter-site difference, APE introduces appearance regulation and provides customized layer-wise aggregation solutions via hypernetworks for each site's personalized parameters. The mutual shape information of instruments is maintained and shared via SGE, which enhances the cross-style shape consistency on the image level and computes the shape-similarity contribution of each site on the prediction level for updating the global parameters. PFedSIS outperforms state-of-the-art methods with +1.51% Dice, +2.11% IoU, -2.79 ASSD, -15.55 HD95 performance gains. The corresponding code and models will be released at https://github.com/wzjialang/PFedSIS.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 71 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

45 of 71 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Segment Anythingcited by 8,158
  7. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  8. 2017Mask R-CNNcited by 7,259
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  10. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  11. 2021Multiscale Vision Transformerscited by 896
  12. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
33 more
  1. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  2. 2021Domain Generalization with MixStylecited by 563
  3. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  4. 2021Towards Personalized Federated Learningcited by 462
  5. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  6. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  7. 2021Personalized Federated Learning using Hypernetworkscited by 210
  8. 2021HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editingcited by 208
  9. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  10. 20192017 Robotic Instrument Segmentation Challengecited by 166
  11. 2018Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentationcited by 148
  12. 20202018 Robotic Scene Segmentation Challengecited by 123
  13. 2023SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentationcited by 103
  14. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 95
  15. 2016Learning feed-forward one-shot learnerscited by 93
  16. 2018Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learningcited by 81
  17. 2020ISINet: An Instance-Based Approach for Surgical Instrument Segmentationcited by 67
  18. 2022Federated Domain Generalization for Image Recognition via Cross-Client Style Transfercited by 40
  19. 2022Hypernetworks in Meta-Reinforcement Learningcited by 39
  20. 2023MATIS: Masked-Attention Transformers for Surgical Instrument Segmentationcited by 36
  21. 2022CD2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learningcited by 34
  22. 2022Federated Learning of Generative Image Priors for MRI Reconstructioncited by 33
  23. 2022Personalizing Federated Medical Image Segmentation via Local Calibrationcited by 28
  24. 2022Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentationcited by 27
  25. 2023Text Promptable Surgical Instrument Segmentation with Vision-Language Modelscited by 27
  26. 2022FedMix: Mixed Supervised Federated Learning for Medical Image Segmentationcited by 22
  27. 2022IOP-FL: Inside-Outside Personalization for Federated Medical Image Segmentationcited by 21
  28. 2021Multi-task Federated Learning for Heterogeneous Pancreas Segmentationcited by 19
  29. 2021One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Videocited by 14
  30. 2021A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Imagescited by 11
  31. 2021Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Datacited by 8
  32. 2022Continual Learning with Dependency Preserving Hypernetworkscited by 6
  33. 2023PPIDSG: A Privacy-Preserving Image Distribution Sharing Scheme with GAN in Federated Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026SurgFusion-Net: Diversified Adaptive Multimodal Fusion Network for Surgical Skill Assessmentcited by 0
  2. 2026SurgFed: Language-guided Multi-Task Federated Learning for Surgical Video Understandingcited by 0