# Movable Antenna Enhanced Federated Fine-Tuning of Large Language Models via Hybrid Client Selection Optimization

arXiv: https://arxiv.org/abs/2506.00011 · HTML: https://mlsanity.com/paper/2506.00011/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation](https://mlsanity.com/paper/1910.13067/) (2019)
- [Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities](https://mlsanity.com/paper/2409.03444/) (2024)
- [SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models](https://mlsanity.com/paper/2407.00952/) (2024)
- [FedAvg with Fine Tuning: Local Updates Lead to Representation Learning](https://mlsanity.com/paper/2205.13692/) (2022)
- [A Survey on Large Language Models with some Insights on their Capabilities and Limitations](https://mlsanity.com/paper/2501.04040/) (2025)
- [Federated Learning for 6G Communications: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/2006.02931/) (2020)
- [Reconfigurable Intelligent Surface Enabled Federated Learning: A Unified Communication-Learning Design Approach](https://mlsanity.com/paper/2011.10282/) (2020)
- [Decentralized Low-Rank Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2501.15361/) (2025)
- [Modeling and Performance Analysis for Movable Antenna Enabled Wireless Communications](https://mlsanity.com/paper/2210.05325/) (2022)
- [Movable-Antenna Enhanced Multiuser Communication via Antenna Position Optimization](https://mlsanity.com/paper/2302.06978/) (2023)
- [A Tutorial on Fluid Antenna System for 6G Networks: Encompassing Communication Theory, Optimization Methods and Hardware Designs](https://mlsanity.com/paper/2407.03449/) (2024)
- [A Tutorial on Movable Antennas for Wireless Networks](https://mlsanity.com/paper/2502.17905/) (2025)
- [SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learning](https://mlsanity.com/paper/2504.18762/) (2025)
- [On Label Shift in Domain Adaptation via Wasserstein Distance](https://mlsanity.com/paper/2110.15520/) (2021)
- [Movable-Antenna Position Optimization: A Graph-based Approach](https://mlsanity.com/paper/2403.16886/) (2024)
- [Movable Antenna-Aided Federated Learning with Over-the-Air Aggregation: Joint Optimization of Positioning, Beamforming, and User Selection](https://mlsanity.com/paper/2411.06721/) (2024)
- [Movable Antennas Meet Intelligent Reflecting Surface: Friends or Foes?](https://mlsanity.com/paper/2501.15880/) (2025)
- [Secure MIMO Communication Relying on Movable Antennas](https://mlsanity.com/paper/2403.04269/) (2024)

## Built on by (most cited first, 1 of 1)

- [Straggler-Resilient Federated Learning over A Hybrid Conventional and Pinching Antenna Network](https://mlsanity.com/paper/2508.15821/) (2025)

## Abstract (arXiv)

Federated fine-tuning of large language models (LLMs) over bandwidth-limited 6G links must meet strict round-time and energy budgets. Analog over-the-air (OTA) aggregation reduces uplink cost but is sensitive to fading and interference, which distort the aggregated gradient. We consider a two-phase workflow (centralized pre-training followed by federated fine-tuning) where the base station uses a movable-antenna (MA) array. In each round, MA element positions and the receive/transmit beamformers are adjusted under minimum-spacing constraints to reshape the channel and improve OTA aggregation without increasing user transmit power. We formulate a mixed-integer, nonconvex resource-allocation problem that jointly selects clients and optimizes the number of global rounds, CPU frequencies, mini-batch sizes, MA positions, and analog weights under end-to-end latency and energy limits. A successive convex approximation-penalty dual decomposition (SCA-PDD) routine alternates convex updates with oblique-manifold beamforming and spacing-aware MA placement. Experiments on OpenLLaMA-v2 (3B) with LoRA and 4-bit quantization on Alpaca and Dolly (10 clients) attain round-30 validation perplexities as low as 2.94 (Alpaca, K=1) and 4.62 (Dolly, K=1). Relative to the strongest non-MA baseline at the same concurrency, this corresponds to 17.4 percent (Alpaca, K=1) and 54.4 percent (Dolly, K=1) lower perplexity; at K=2 the reductions are 14.2 percent (Alpaca) and 13.7 percent (Dolly). Participation fairness also improves across all uplink concurrencies K in {1,2,4,8}, with the largest margins when fewer clients transmit per round.