# FedLPPA: Learning Personalized Prompt and Aggregation for Federated Weakly-supervised Medical Image Segmentation arXiv: https://arxiv.org/abs/2402.17502 ยท HTML: https://mlsanity.com/paper/2402.17502/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 46 of its 66 references that are in the map. - First posted on arXiv 27 Feb 2024. ## Builds on (most cited first, 30 of 46) - [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016) - [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018) - [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021) - [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022) - [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019) - [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018) - [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019) - [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021) - [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021) - [Towards Personalized Federated Learning](https://mlsanity.com/paper/2103.00710/) (2021) - [clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation](https://mlsanity.com/paper/2003.07311/) (2020) - [FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space](https://mlsanity.com/paper/2103.06030/) (2021) - [Exploring Visual Prompts for Adapting Large-Scale Models](https://mlsanity.com/paper/2203.17274/) (2022) - [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019) - [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023) - [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016) - [Medical Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2009.13120/) (2020) - [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019) - [SAM-Med2D](https://mlsanity.com/paper/2308.16184/) (2023) - [FedALA: Adaptive Local Aggregation for Personalized Federated Learning](https://mlsanity.com/paper/2212.01197/) (2022) - [MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data](https://mlsanity.com/paper/2002.03366/) (2020) - [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018) - [ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model](https://mlsanity.com/paper/2007.05201/) (2020) - [CycleMix: A Holistic Strategy for Medical Image Segmentation from Scribble Supervision](https://mlsanity.com/paper/2203.01475/) (2022) - [Scribble-Supervised Medical Image Segmentation via Dual-Branch Network and Dynamically Mixed Pseudo Labels Supervision](https://mlsanity.com/paper/2203.02106/) (2022) - [OCTA-500: A Retinal Dataset for Optical Coherence Tomography Angiography Study](https://mlsanity.com/paper/2012.07261/) (2020) - [Tree Energy Loss: Towards Sparsely Annotated Semantic Segmentation](https://mlsanity.com/paper/2203.10739/) (2022) - [UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learner](https://mlsanity.com/paper/2304.03493/) (2023) - [GAMMA Challenge:Glaucoma grAding from Multi-Modality imAges](https://mlsanity.com/paper/2202.06511/) (2022) - [Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching](https://mlsanity.com/paper/2106.08600/) (2021) ## Built on by (most cited first, 6 of 6) - [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025) - [Rethinking Membership Inference Attacks Against Transfer Learning](https://mlsanity.com/paper/2501.11577/) (2025) - [Weakly Supervised Segmentation Framework for Thyroid Nodule Based on High-confidence Labels and High-rationality Losses](https://mlsanity.com/paper/2502.19707/) (2025) - [Prompt Mechanisms in Medical Imaging: A Comprehensive Survey](https://mlsanity.com/paper/2507.01055/) (2025) - [ZScribbleSeg: A comprehensive segmentation framework with modeling of efficient annotation and maximization of scribble supervision](https://mlsanity.com/paper/2605.06266/) (2026) - [MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities](https://mlsanity.com/paper/2608.19788/) (2026) ## Abstract (arXiv) Federated learning (FL) effectively mitigates the data silo challenge brought about by policies and privacy concerns, implicitly harnessing more data for deep model training. However, traditional centralized FL models grapple with diverse multi-center data, especially in the face of significant data heterogeneity, notably in medical contexts. In the realm of medical image segmentation, the growing imperative to curtail annotation costs has amplified the importance of weakly-supervised techniques which utilize sparse annotations such as points, scribbles, etc. A pragmatic FL paradigm shall accommodate diverse annotation formats across different sites, which research topic remains under-investigated. In such context, we propose a novel personalized FL framework with learnable prompt and aggregation (FedLPPA) to uniformly leverage heterogeneous weak supervision for medical image segmentation. In FedLPPA, a learnable universal knowledge prompt is maintained, complemented by multiple learnable personalized data distribution prompts and prompts representing the supervision sparsity. Integrated with sample features through a dual-attention mechanism, those prompts empower each local task decoder to adeptly adjust to both the local distribution and the supervision form. Concurrently, a dual-decoder strategy, predicated on prompt similarity, is introduced for enhancing the generation of pseudo-labels in weakly-supervised learning, alleviating overfitting and noise accumulation inherent to local data, while an adaptable aggregation method is employed to customize the task decoder on a parameter-wise basis. Extensive experiments on four distinct medical image segmentation tasks involving different modalities underscore the superiority of FedLPPA, with its efficacy closely parallels that of fully supervised centralized training. Our code and data will be available.