mlsanityData to 14 Oct 2026

A Survey on Federated Fine-tuning of Large Language Models

arXiv 2503.12016 ↗First posted 15 Mar 2025cs.LG cs.DCcited by 24

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive success across various tasks. Integrating LLMs with Federated Learning (FL), a paradigm known as FedLLM, offers a promising avenue for collaborative model adaptation while preserving data privacy. This survey provides a systematic and comprehensive review of FedLLM. We begin by tracing the historical development of both LLMs and FL, summarizing relevant prior research to set the context. Subsequently, we delve into an in-depth analysis of the fundamental challenges inherent in deploying FedLLM. Addressing these challenges often requires efficient adaptation strategies; therefore, we conduct an extensive examination of existing Parameter-Efficient Fine-tuning (PEFT) methods and explore their applicability within the FL framework. To rigorously evaluate the performance of FedLLM, we undertake a thorough review of existing fine-tuning datasets and evaluation benchmarks. Furthermore, we discuss FedLLM's diverse real-world applications across multiple domains. Finally, we identify critical open challenges and outline promising research directions to foster future advancements in FedLLM. This survey aims to serve as a foundational resource for researchers and practitioners, offering valuable insights into the rapidly evolving landscape of federated fine-tuning for LLMs. It also establishes a roadmap for future innovations in privacy-preserving AI. We actively maintain a hrefhttps://github.com/Clin0212/Awesome-Federated-LLM-LearningGitHub repo to track cutting-edge advancements in this field.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Mar 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 17.

24 papers.

Sort:

By area:

  1. 2026Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalizationcited by 5
  2. 2026Revisiting Backdoor Threat in Federated Instruction Tuning from a Signal Aggregation Perspectivecited by 5
  3. 2025Learning Like Humans: Resource-Efficient Federated Fine-Tuning through Cognitive Developmental Stagescited by 4
  4. 2025Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimizationcited by 3
  5. 2025Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Surveycited by 2
  6. 2026Rethinking LoRA for Data Heterogeneous Federated Learning: Subspace and State Alignmentcited by 2
  7. 2026ProtegoFed: Backdoor-Free Federated Instruction Tuning with Interspersed Poisoned Datacited by 2
  8. 2025Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Modelscited by 1
  9. 2025FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Modelscited by 1
  10. 2026Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factorcited by 1
  11. 2026EmbTracker: Traceable Black-box Watermarking for Federated Language Modelscited by 1
  12. 2026APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMscited by 1
12 more
  1. 2026Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMscited by 1
  2. 2025FLAME: Towards Federated Fine-Tuning Large Language Models Through Adaptive SMoEcited by 0
  3. 2025BadPromptFL: A Novel Backdoor Threat to Prompt-based Federated Learning in Multimodal Modelscited by 0
  4. 2026Resource-Adaptive Federated Text Generation with Differential Privacycited by 0
  5. 2026FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuningcited by 0
  6. 2026PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMscited by 0
  7. 2026FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecastingcited by 0
  8. 2026Thermo-FL: Thermal-Aware Robust Federated Fine-Tuning of Large Language Models for Edge AIcited by 0
  9. 2026CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimizationcited by 0
  10. 2026CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memorycited by 0
  11. 2026HO-FL: Hybrid-Order Federated Learning for Heterogeneous Edge Devicescited by 0
  12. 2026FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantizationcited by 0