# Federated Fine-Tuning for Pre-Trained Foundation Models Over Wireless Networks

arXiv: https://arxiv.org/abs/2407.02924 · HTML: https://mlsanity.com/paper/2407.02924/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2024.

## Built on by (most cited first, 25 of 25)

- [An overview of domain-specific foundation model: key technologies, applications and challenges](https://mlsanity.com/paper/2409.04267/) (2024)
- [Federated Low-Rank Adaptation for Foundation Models: A Survey](https://mlsanity.com/paper/2505.13502/) (2025)
- [Resource Management for Low-latency Cooperative Fine-tuning of Foundation Models at the Network Edge](https://mlsanity.com/paper/2407.09873/) (2024)
- [Energy-Efficient Split Learning for Fine-Tuning Large Language Models in Edge Networks](https://mlsanity.com/paper/2412.00090/) (2024)
- [SplitFrozen: Split Learning with Device-side Model Frozen for Fine-Tuning LLM on Heterogeneous Resource-Constrained Devices](https://mlsanity.com/paper/2503.18986/) (2025)
- [FedHL: Federated Learning for Heterogeneous Low-Rank Adaptation via Unbiased Aggregation](https://mlsanity.com/paper/2505.18494/) (2025)
- [Graph Representation-based Model Poisoning on the Heterogeneous Internet of Agents](https://mlsanity.com/paper/2511.07176/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Edge Large AI Models: Revolutionizing 6G Networks](https://mlsanity.com/paper/2505.00321/) (2025)
- [Two Stage Wireless Federated LoRA Fine-Tuning with Sparsified Orthogonal Updates](https://mlsanity.com/paper/2505.00333/) (2025)
- [Tool-Aided Evolutionary LLM for Generative Policy Toward Efficient Resource Management in Wireless Federated Learning](https://mlsanity.com/paper/2505.11570/) (2025)
- [FedLoDrop: Federated LoRA with Dropout for Generalized LLM Fine-tuning](https://mlsanity.com/paper/2510.12078/) (2025)
- [Federated Low-Rank Adaptation with Differential Privacy over Wireless Networks](https://mlsanity.com/paper/2411.07806/) (2024)
- [FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA](https://mlsanity.com/paper/2506.01194/) (2025)
- [Generative AI Meets Wireless Sensing: Towards Wireless Foundation Model](https://mlsanity.com/paper/2509.15258/) (2025)
- [CollaPipe: Adaptive Segment-Optimized Pipeline Parallelism for Collaborative LLM Training in Heterogeneous Edge Networks](https://mlsanity.com/paper/2509.19855/) (2025)
- [ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning](https://mlsanity.com/paper/2603.04437/) (2026)
- [Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs](https://mlsanity.com/paper/2605.07961/) (2026)
- [Satellite Federated Fine-Tuning for Foundation Models in Space Computing Power Networks](https://mlsanity.com/paper/2504.10403/) (2025)
- [ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation](https://mlsanity.com/paper/2603.04436/) (2026)
- [FedLNS: Leverage LayerNorm Signature Modeling to Mitigate Adversarial Manipulation in Federated LLMs](https://mlsanity.com/paper/2608.18736/) (2026)
- [SplitLite: Low-Rank Residual Compression for Split Learning](https://mlsanity.com/paper/2608.23018/) (2026)
- [Task-Aware Federated Fine-Tuning for MoE-based Large Language Models](https://mlsanity.com/paper/2609.13395/) (2026)
- [FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization](https://mlsanity.com/paper/2610.01537/) (2026)
- [Byzantine-Resilient Federated Large Language Model via Hypergraph Message Passing](https://mlsanity.com/paper/2610.12711/) (2026)

## Abstract (arXiv)

Pre-trained foundation models (FMs), with extensive number of neurons, are key to advancing next-generation intelligence services, where personalizing these models requires massive amount of task-specific data and computational resources. The prevalent solution involves centralized processing at the edge server, which, however, raises privacy concerns due to the transmission of raw data. Instead, federated fine-tuning (FedFT) is an emerging privacy-preserving fine-tuning (FT) paradigm for personalized pre-trained foundation models. In particular, by integrating low-rank adaptation (LoRA) with federated learning (FL), federated LoRA enables the collaborative FT of a global model with edge devices, achieving comparable learning performance to full FT while training fewer parameters over distributed data and preserving raw data privacy. However, the limited radio resources and computation capabilities of edge devices pose significant challenges for deploying federated LoRA over wireless networks. To this paper, we propose a split federated LoRA framework, which deploys the computationally-intensive encoder of a pre-trained model at the edge server, while keeping the embedding and task modules at the edge devices. Building on this split framework, the paper provides a rigorous analysis of the upper bound of the convergence gap for the wireless federated LoRA system. This analysis motivates the formulation of a long-term upper bound minimization problem, where we decompose the formulated long-term mixed-integer programming (MIP) problem into sequential sub-problems using the Lyapunov technique. We then develop an online algorithm for effective device scheduling and bandwidth allocation. Simulation results demonstrate the effectiveness of the proposed online algorithm in enhancing learning performance.