Movable Antenna Enhanced Federated Fine-Tuning of Large Language Models via Hybrid Client Selection Optimization
Not on the map. Why?
Abstract (arXiv)
Federated fine-tuning of large language models (LLMs) over bandwidth-limited 6G links must meet strict round-time and energy budgets. Analog over-the-air (OTA) aggregation reduces uplink cost but is sensitive to fading and interference, which distort the aggregated gradient. We consider a two-phase workflow (centralized pre-training followed by federated fine-tuning) where the base station uses a movable-antenna (MA) array. In each round, MA element positions and the receive/transmit beamformers are adjusted under minimum-spacing constraints to reshape the channel and improve OTA aggregation without increasing user transmit power. We formulate a mixed-integer, nonconvex resource-allocation problem that jointly selects clients and optimizes the number of global rounds, CPU frequencies, mini-batch sizes, MA positions, and analog weights under end-to-end latency and energy limits. A successive convex approximation-penalty dual decomposition (SCA-PDD) routine alternates convex updates with oblique-manifold beamforming and spacing-aware MA placement. Experiments on OpenLLaMA-v2 (3B) with LoRA and 4-bit quantization on Alpaca and Dolly (10 clients) attain round-30 validation perplexities as low as 2.94 (Alpaca, K=1) and 4.62 (Dolly, K=1). Relative to the strongest non-MA baseline at the same concurrency, this corresponds to 17.4 percent (Alpaca, K=1) and 54.4 percent (Dolly, K=1) lower perplexity; at K=2 the reductions are 14.2 percent (Alpaca) and 13.7 percent (Dolly). Participation fairness also improves across all uplink concurrencies K in {1,2,4,8}, with the largest margins when fewer clients transmit per round.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 16 May 2025.
Builds on
32 of 49 references.
References · 32
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2020In Search of Lost Domain Generalizationcited by 1,054
- 2018Local SGD Converges Fast and Communicates Littlecited by 605
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
- 2021Differentially Private Fine-tuning of Language Modelscited by 400
- 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
- 2018Federated Learning via Over-the-Air Computationcited by 219
- 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
- 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
20 more
- 2019Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocationcited by 94
- 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiescited by 69
- 2024SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Modelscited by 62
- 2022FedAvg with Fine Tuning: Local Updates Lead to Representation Learningcited by 56
- 2025A Survey on Large Language Models with some Insights on their Capabilities and Limitationscited by 42
- 2020Federated Learning for 6G Communications: Challenges, Methods, and Future Directionscited by 33
- 2020Reconfigurable Intelligent Surface Enabled Federated Learning: A Unified Communication-Learning Design Approachcited by 33
- 2025Decentralized Low-Rank Fine-Tuning of Large Language Modelscited by 14
- 2022Modeling and Performance Analysis for Movable Antenna Enabled Wireless Communicationscited by 6
- 2023Movable-Antenna Enhanced Multiuser Communication via Antenna Position Optimizationcited by 4
- 2024A Tutorial on Fluid Antenna System for 6G Networks: Encompassing Communication Theory, Optimization Methods and Hardware Designscited by 4
- 2025A Tutorial on Movable Antennas for Wireless Networkscited by 4
- 2025SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learningcited by 4
- 2021On Label Shift in Domain Adaptation via Wasserstein Distancecited by 3
- 2024Movable-Antenna Position Optimization: A Graph-based Approachcited by 3
- 2024Movable Antenna-Aided Federated Learning with Over-the-Air Aggregation: Joint Optimization of Positioning, Beamforming, and User Selectioncited by 3
- 2025Movable Antennas Meet Intelligent Reflecting Surface: Friends or Foes?cited by 3
- 2024Secure MIMO Communication Relying on Movable Antennascited by 1
- 2024Movable Antenna-enabled RIS-aided Integrated Sensing and Communicationcited by 1
- 2024Movable Antennas: Channel Measurement, Modeling, and Performance Evaluationcited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025FedQuad: Adaptive Layer-wise LoRA Deployment and Activation Quantization for Federated Fine-Tuning
- 2026FedSubMuon: Communication-Efficient Federated LLM Fine-Tuning via Structured Subspace Muon
- 2025Split Fine-Tuning for Large Language Models in Wireless Networks
- 2025Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
- 2026CFLoRA: Federated Fine-tuning of LLMs with Complementary Factors for Error-free Aggregation
- 2026FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
- 2025Quantized Analog Beamforming Enabled Multi-task Federated Learning Over-the-air
- 2026AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environments