mlsanityData to 7 Oct 2026

Dual-Personalizing Adapter for Federated Foundation Models

arXiv 2403.19211 ↗First posted 28 Mar 2024cs.LG cs.AI cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

Recently, foundation models, particularly large language models (LLMs), have demonstrated an impressive ability to adapt to various tasks by fine-tuning diverse instruction data. Notably, federated foundation models (FedFM) emerge as a privacy preservation method to fine-tune models collaboratively under federated learning (FL) settings by leveraging many distributed datasets with non-IID data. To alleviate communication and computation overhead, parameter-efficient methods are introduced for efficiency, and some research adapted personalization methods to FedFM for better user preferences alignment. However, a critical gap in existing research is the neglect of test-time distribution shifts in real-world applications, and conventional methods for test-time distribution shifts in personalized FL are less effective for FedFM due to their failure to adapt to complex distribution shift scenarios and the requirement to train all parameters. To bridge this gap, we refine the setting in FedFM, termed test-time personalization, which aims to learn personalized federated foundation models on clients while effectively handling test-time distribution shifts simultaneously. To address challenges in this setting, we explore a simple yet effective solution, a Federated Dual-Personalizing Adapter (FedDPA) architecture. By co-working with a foundation model, a global adapter and a local adapter jointly tackle the test-time distribution shifts and client-specific personalization. Additionally, we introduce an instance-wise dynamic weighting mechanism that dynamically integrates the global and local adapters for each test instance during inference, facilitating effective test-time personalization. The effectiveness of the proposed method has been evaluated on benchmark datasets across different NLP tasks.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

36 of 45 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  12. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
24 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2021Towards Personalized Federated Learningcited by 462
  3. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  4. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  5. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  6. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  7. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 158
  8. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  9. 2022KronA: Parameter Efficient Tuning with Kronecker Adaptercited by 130
  10. 2022Federated Class-Incremental Learningcited by 126
  11. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 92
  12. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  13. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 74
  14. 2021Out of Distribution Generalization in Machine Learningcited by 59
  15. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 59
  16. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
  17. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  18. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  19. 2024DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrievalcited by 26
  20. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 25
  21. 2023FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Modelscited by 25
  22. 2023Adaptive Test-Time Personalization for Federated Learningcited by 23
  23. 2023Low-Parameter Federated Learning with Large Language Modelscited by 22
  24. 2024EVA: Zero-shot Accurate Attributes and Multi-Object Video Editingcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 27; 2026: 21.

54 papers.

Sort:

By area:

  1. 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
  2. 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
  3. 2025Ten Challenging Problems in Federated Foundation Modelscited by 20
  4. 2025A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learningcited by 15
  5. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  6. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRAcited by 13
  7. 2024Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architecturescited by 10
  8. 2024Personalized Federated Collaborative Filtering: A Variational AutoEncoder Approachcited by 9
  9. 2025Vision Foundation Models in Medical Image Analysis: Advances and Challengescited by 9
  10. 2025Exploring the Vulnerabilities of Federated Learning: A Deep Dive into Gradient Inversion Attackscited by 9
  11. 2025Federated Low-Rank Adaptation for Foundation Models: A Surveycited by 9
  12. 2025Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuningcited by 5
38 more
  1. 2025FedSRD: Sparsify-Reconstruct-Decompose for Communication-Efficient Federated Large Language Models Fine-Tuningcited by 5
  2. 2026FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignmentcited by 5
  3. 2025FedMerge: Federated Personalization via Model Mergingcited by 4
  4. 2026Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approachcited by 3
  5. 2025Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptationcited by 2
  6. 2025Federated Adapter on Foundation Models: An Out-Of-Distribution Approachcited by 2
  7. 2025A Survey on Foundation Models for Personalized Federated Intelligencecited by 2
  8. 2025FLEx: Personalized Federated Learning for Mixture-of-Experts LLMs via Expert Graftingcited by 2
  9. 2025Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MRcited by 2
  10. 2025APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squarescited by 2
  11. 2025TAP: Two-Stage Adaptive Personalization of Multi-Task and Multi-Modal Foundation Models in Federated Learningcited by 2
  12. 2026FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuningcited by 2
  13. 2024Sequential Compression Layers for Efficient Federated Learning in Foundational Modelscited by 1
  14. 2025FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Modelscited by 1
  15. 2025FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregationcited by 1
  16. 2025Foundational Models and Federated Learning: Survey, Taxonomy, Challenges and Practical Insightscited by 1
  17. 2025FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraintscited by 1
  18. 2025Towards Continuous Intelligence Growth: Self-Training, Continual Learning, and Dual-Scale Memory in SuperIntelliAgentcited by 1
  19. 2026SDFLoRA: Selective Decoupled Federated LoRA for Privacy-preserving Fine-tuning with Heterogeneous Clientscited by 1
  20. 2026Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factorcited by 1
  21. 2026Task-Centric Personalized Federated Fine-Tuning of Language Modelscited by 1
  22. 2026FedPA-LoRA: Product-Aligned Framework for Mitigating Aggregation and Initialization Errors in Heterogeneous Federated LoRAcited by 1
  23. 2024Personalized Additive Modeling for Multi-level Federated Learningcited by 0
  24. 2025A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendationcited by 0
  25. 2025Bringing Multi-Modal Multi-Task Federated Foundation Models to Education Domain: Prospects and Challengescited by 0
  26. 2025Bi-level Personalization for Federated Foundation Models: A Task-vector Aggregation Approachcited by 0
  27. 2025Communication-Efficient and Accurate Approach for Aggregation in Federated Low-Rank Adaptationcited by 0
  28. 2025Personalized Federated Fine-Tuning of Vision Foundation Models for Healthcarecited by 0
  29. 2026Collaborative and Efficient Fine-tuning: Leveraging Task Similaritycited by 0
  30. 2026The Subjectivity of Respect in Police Traffic Stops: Modeling Community Perspectives in Body-Worn Camera Footagecited by 0
  31. 2026SecureGate: Learning When to Reveal PII Safely via Token-Gated Dual-Adapters for Federated LLMscited by 0
  32. 2026Communication-Efficient Personalized Adaptation via Federated-Local Model Mergingcited by 0
  33. 2026HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learningcited by 0
  34. 2026Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?cited by 0
  35. 2026Silent Failures in Federated Personalization of Foundation Modelscited by 0
  36. 2026Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentationcited by 0
  37. 2026Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approachcited by 0
  38. 2026Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneitycited by 0

50 most cited of 54 All 54 (JSON)