mlsanityData to 14 Oct 2026

FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models

arXiv 2506.14824 ↗First posted 12 Jun 2025cs.LG cs.AI cs.MMcited by 1

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) excel in tasks like multimodal reasoning and cross-modal retrieval but face deployment challenges in real-world scenarios due to distributed multimodal data and strict privacy requirements. Federated Learning (FL) offers a solution by enabling collaborative model training without centralizing data. However, realizing FL for MLLMs presents significant challenges, including high computational demands, limited client capacity, substantial communication costs, and heterogeneous client data. Existing FL methods assume client-side deployment of full models, an assumption that breaks down for large-scale MLLMs due to their massive size and communication demands. To address these limitations, we propose FedNano, the first FL framework that centralizes the LLM on the server while introducing NanoEdge, a lightweight module for client-specific adaptation. NanoEdge employs modality-specific encoders, connectors, and trainable NanoAdapters with low-rank adaptation. This design eliminates the need to deploy LLM on clients, reducing client-side storage by 95%, and limiting communication overhead to only 0.01% of the model parameters. By transmitting only compact NanoAdapter updates, FedNano handles heterogeneous client data and resource constraints while preserving privacy. Experiments demonstrate that FedNano outperforms prior FL baselines, bridging the gap between MLLM scale and FL feasibility, and enabling scalable, decentralized multimodal AI systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 35 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

18 of 35 references.

References · 18

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2023Instruction Tuning with GPT-4cited by 686
  10. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
  11. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  12. 2023Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimizationcited by 65
6 more
  1. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
  2. 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 55
  3. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  4. 2023pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuningcited by 40
  5. 2024Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modalitycited by 14
  6. 2024FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Datacited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Step Toward Federated Pretraining of Multimodal Large Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
  2. 2026FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks
  3. 2025FedSODA: Federated Fine-tuning of LLMs via Similarity Group Pruning and Orchestrated Distillation Alignment
  4. 2024Open-Vocabulary Federated Learning with Multimodal Prototyping
  5. 2024Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
  6. 2025Federated Large Language Models: Feasibility, Robustness, Security and Future Directions
  7. 2025FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-World LoRA
  8. 2024HAFLQ: Heterogeneous Adaptive Federated LoRA Fine-tuned LLM with Quantization