A Hybrid Swarm Intelligence Approach for Optimizing Multimodal Large Language Models Deployment in Edge-Cloud-based Federated Learning Environments
Not on the map. Why?
Abstract (arXiv)
The combination of Federated Learning (FL), Multimodal Large Language Models (MLLMs), and edge-cloud computing enables distributed and real-time data processing while preserving privacy across edge devices and cloud infrastructure. However, the deployment of MLLMs in FL environments with resource-constrained edge devices presents significant challenges, including resource management, communication overhead, and non-IID data. To address these challenges, we propose a novel hybrid framework wherein MLLMs are deployed on edge devices equipped with sufficient resources and battery life, while the majority of training occurs in the cloud. To identify suitable edge devices for deployment, we employ Particle Swarm Optimization (PSO), and Ant Colony Optimization (ACO) is utilized to optimize the transmission of model updates between edge and cloud nodes. This proposed swarm intelligence-based framework aims to enhance the efficiency of MLLM training by conducting extensive training in the cloud and fine-tuning at the edge, thereby reducing energy consumption and communication costs. Our experimental results show that the proposed method significantly improves system performance, achieving an accuracy of 92%, reducing communication cost by 30%, and enhancing client participation compared to traditional FL methods. These results make the proposed approach highly suitable for large-scale edge-cloud computing systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 72 references that are in the map. First posted on arXiv 4 Feb 2025.
Builds on
34 of 72 references.
References · 34
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
- 2023Detecting Pretraining Data from Large Language Modelscited by 387
- 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
- 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 232
- 2023Multimodal Large Language Models: A Surveycited by 195
- 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 149
- 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
- 2020Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehiclescited by 110
- 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 97
- 2024FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Modelcited by 74
22 more
- 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 71
- 2024WorldGPT: Empowering LLM as Multimodal World Modelcited by 42
- 2023Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learningcited by 41
- 2024Federated Large Language Models: Current Progress and Future Directionscited by 27
- 2024Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligencecited by 27
- 2023LLMGA: Multimodal Large Language Model based Generation Assistantcited by 25
- 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
- 2023Low-Parameter Federated Learning with Large Language Modelscited by 23
- 2019MULE: Multimodal Universal Language Embeddingcited by 22
- 2024An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Modelscited by 22
- 2023FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUscited by 20
- 2018Attention-based Audio-Visual Fusion for Robust Automatic Speech Recognitioncited by 18
- 2024pFedLVM: A Large Vision Model (LVM)-Driven and Latent Feature-Based Personalized Federated Learning Framework in Autonomous Drivingcited by 16
- 2024AlignGPT: Multi-modal Large Language Models with Adaptive Alignment Capabilitycited by 15
- 2024Multimodal LLM Enhanced Cross-lingual Cross-modal Retrievalcited by 13
- 2024Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Frameworkcited by 11
- 2024SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimizationcited by 11
- 2024Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approachescited by 6
- 2024A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Modelscited by 5
- 2024Enhancing IoT Intelligence: A Transformer-based Reinforcement Learning Methodologycited by 4
- 2023Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applicationscited by 3
- 2024Leveraging Foundation Models for Efficient Federated Learning in Resource-restricted Edge Networkscited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Emerging Paradigms for Securing Federated Learning Systems25 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
- 2021An Optimization Framework for Federated Edge Learning
- 2024Leveraging Federated Learning and Edge Computing for Recommendation Systems within Cloud Computing Networks
- 2026CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization
- 2026AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environments
- 2025Enhancing QoS in Edge Computing through Federated Layering Techniques: A Pathway to Resilient AI Lifelong Learning Systems
- 2025FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
- 2026Federated Prompt Learning: A Unified Framework, Empirical Analysis, and Future Directions