mlsanityData to 7 Oct 2026

FedPFT: Federated Proxy Fine-Tuning of Foundation Models

arXiv 2404.11536 ↗First posted 17 Apr 2024cs.LG cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Adapting Foundation Models (FMs) for downstream tasks through Federated Learning (FL) emerges a promising strategy for protecting data privacy and valuable FMs. Existing methods fine-tune FM by allocating sub-FM to clients in FL, however, leading to suboptimal performance due to insufficient tuning and inevitable error accumulations of gradients. In this paper, we propose Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation in downstream tasks through FL by two key modules. First, the sub-FM construction module employs a layer-wise compression approach, facilitating comprehensive FM fine-tuning across all layers by emphasizing those crucial neurons. Second, the sub-FM alignment module conducts a two-step distillations-layer-level and neuron-level-before and during FL fine-tuning respectively, to reduce error of gradient by accurately aligning sub-FM with FM under theoretical guarantees. Experimental results on seven commonly used datasets (i.e., four text and three vision) demonstrate the superiority of FedPFT.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

31 of 37 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
19 more
  1. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  2. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  4. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  5. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  6. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  7. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  8. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  9. 2020On the Effect of Dropping Layers of Pre-trained Transformer Modelscited by 118
  10. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  11. 2023Offsite-Tuning: Transfer Learning without Full Modelcited by 77
  12. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 74
  13. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 62
  14. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
  15. 2023Integration of Large Language Models and Federated Learningcited by 28
  16. 2022Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Trainingcited by 24
  17. 2022FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learningcited by 17
  18. 2023FedBone: Towards Large-Scale Federated Multi-Task Learningcited by 14
  19. 2023FLGo: A Fully Customizable Federated Learning Platformcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Synergizing Foundation Models and Federated Learning: A Surveycited by 14
  2. 2025Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directionscited by 10
  3. 2024FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injectioncited by 8
  4. 2024Federated In-Context LLM Agent Learningcited by 8
  5. 2024FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation Modelscited by 5
  6. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  7. 2024Personalized Quantum Federated Learning for Privacy Image Classificationcited by 4
  8. 2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
  9. 2026FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuningcited by 3
  10. 2025FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCAcited by 1
  11. 2025A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systemscited by 1
  12. 2025HFedCKD: Toward Robust Heterogeneous Federated Learning via Data-free Knowledge Distillation and Two-way Contrastcited by 0
4 more
  1. 2025FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignmentcited by 0
  2. 2025On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility Viewcited by 0
  3. 2025FedMMKT:Co-Enhancing a Server Text-to-Image Model and Client Task Models in Multi-Modal Federated Learningcited by 0
  4. 2025Implicit Federated In-context Learning For Task-Specific LLM Fine-Tuningcited by 0