Inclusive Data Representation in Federated Learning: A Novel Approach Integrating Textual and Visual Prompt
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) is often impeded by communication overhead issues. Prompt tuning, as a potential solution, has been introduced to only adjust a few trainable parameters rather than the whole model. However, current single-modality prompt tuning approaches fail to comprehensively portray local clients' data. To overcome this limitation, we present Twin Prompt Federated learning (TPFL), a pioneering solution that integrates both visual and textual modalities, ensuring a more holistic representation of local clients' data characteristics. Furthermore, in order to tackle the data heterogeneity issues, we introduce the Augmented TPFL (ATPFL) employing the contrastive learning to TPFL, which not only enhances the global knowledge acquisition of client models but also fosters the development of robust, compact models. The effectiveness of TPFL and ATPFL is substantiated by our extensive evaluations, consistently showing superior performance compared to all baselines.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 4 Oct 2023.
Builds on
22 of 29 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
- 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
- 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
- 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
10 more
- 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 133
- 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
- 2023Learning Federated Visual Prompt in Null Space for MRI Reconstructioncited by 40
- 2019A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networkscited by 36
- 2020Communication-Efficient Federated Distillationcited by 28
- 2020Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospectivecited by 27
- 2022Towards Efficient Communications in Federated Learning: A Contemporary Surveycited by 24
- 2022SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communicationscited by 11
- 2020Sparse Communication for Training Deep Networkscited by 10
- 2023AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategycited by 7
Built on by
Citing papers per year: 2024: 1.
1 paper.