# A Survey on Federated Fine-tuning of Large Language Models

arXiv: https://arxiv.org/abs/2503.12016 · HTML: https://mlsanity.com/paper/2503.12016/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2025.

## Built on by (most cited first, 24 of 24)

- [Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalization](https://mlsanity.com/paper/2602.15028/) (2026)
- [Revisiting Backdoor Threat in Federated Instruction Tuning from a Signal Aggregation Perspective](https://mlsanity.com/paper/2602.15671/) (2026)
- [Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stages](https://mlsanity.com/paper/2508.00041/) (2025)
- [Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization](https://mlsanity.com/paper/2506.02370/) (2025)
- [Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey](https://mlsanity.com/paper/2510.00078/) (2025)
- [Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignment](https://mlsanity.com/paper/2602.01746/) (2026)
- [ProtegoFed: Backdoor-Free Federated Instruction Tuning with Interspersed Poisoned Data](https://mlsanity.com/paper/2603.00516/) (2026)
- [Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2506.06060/) (2025)
- [FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Models](https://mlsanity.com/paper/2508.10020/) (2025)
- [Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor](https://mlsanity.com/paper/2603.08058/) (2026)
- [EmbTracker: Traceable Black-box Watermarking for Federated Language Models](https://mlsanity.com/paper/2603.12089/) (2026)
- [APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMs](https://mlsanity.com/paper/2604.04261/) (2026)
- [Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs](https://mlsanity.com/paper/2605.07961/) (2026)
- [FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoE](https://mlsanity.com/paper/2506.16600/) (2025)
- [BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Models](https://mlsanity.com/paper/2508.08040/) (2025)
- [Resource-Adaptive Federated Text Generation with Differential Privacy](https://mlsanity.com/paper/2603.07027/) (2026)
- [FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning](https://mlsanity.com/paper/2605.06596/) (2026)
- [PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs](https://mlsanity.com/paper/2607.12111/) (2026)
- [FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecasting](https://mlsanity.com/paper/2608.01290/) (2026)
- [Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AI](https://mlsanity.com/paper/2608.21172/) (2026)
- [CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization](https://mlsanity.com/paper/2609.15664/) (2026)
- [CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory](https://mlsanity.com/paper/2609.36935/) (2026)
- [HO-FL: Hybrid-Order Federated Learning for Heterogeneous Edge Devices](https://mlsanity.com/paper/2609.39074/) (2026)
- [FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization](https://mlsanity.com/paper/2610.01537/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive success across various tasks. Integrating LLMs with Federated Learning (FL), a paradigm known as FedLLM, offers a promising avenue for collaborative model adaptation while preserving data privacy. This survey provides a systematic and comprehensive review of FedLLM. We begin by tracing the historical development of both LLMs and FL, summarizing relevant prior research to set the context. Subsequently, we delve into an in-depth analysis of the fundamental challenges inherent in deploying FedLLM. Addressing these challenges often requires efficient adaptation strategies; therefore, we conduct an extensive examination of existing Parameter-Efficient Fine-tuning (PEFT) methods and explore their applicability within the FL framework. To rigorously evaluate the performance of FedLLM, we undertake a thorough review of existing fine-tuning datasets and evaluation benchmarks. Furthermore, we discuss FedLLM's diverse real-world applications across multiple domains. Finally, we identify critical open challenges and outline promising research directions to foster future advancements in FedLLM. This survey aims to serve as a foundational resource for researchers and practitioners, offering valuable insights into the rapidly evolving landscape of federated fine-tuning for LLMs. It also establishes a roadmap for future innovations in privacy-preserving AI. We actively maintain a hrefhttps://github.com/Clin0212/Awesome-Federated-LLM-LearningGitHub repo to track cutting-edge advancements in this field.