Multimodal Online Federated Learning with Modality Missing in Internet of Things
Not on the map. Why?
Abstract (arXiv)
The Internet of Things (IoT) ecosystem generates vast amounts of multimodal data from heterogeneous sources such as sensors, cameras, and microphones. As edge intelligence continues to evolve, IoT devices have progressed from simple data collection units to nodes capable of executing complex computational tasks. This evolution necessitates the adoption of distributed learning strategies to effectively handle multimodal data in an IoT environment. Furthermore, the real-time nature of data collection and limited local storage on edge devices in IoT call for an online learning paradigm. To address these challenges, we introduce the concept of Multimodal Online Federated Learning (MMO-FL), a novel framework designed for dynamic and decentralized multimodal learning in IoT environments. Building on this framework, we further account for the inherent instability of edge devices, which frequently results in missing modalities during the learning process. We conduct a comprehensive theoretical analysis under both complete and missing modality scenarios, providing insights into the performance degradation caused by missing modalities. To mitigate the impact of modality missing, we propose the Prototypical Modality Mitigation (PMM) algorithm, which leverages prototype learning to effectively compensate for missing modalities. Experimental results on two multimodal datasets further demonstrate the superior performance of PMM compared to benchmarks.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 22 May 2025.
Builds on
20 of 41 references.
References · 20
- 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 408
- 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
- 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
- 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
- 2017Online Deep Learning: Learning Deep Neural Networks on the Flycited by 86
- 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 72
- 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
- 2023FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Updatecited by 32
- 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
- 2023Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentationcited by 24
- 2024Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modalitycited by 21
- 2023Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrastcited by 17
8 more
- 2022Tighter Regret Analysis and Optimization of Online Federated Learningcited by 14
- 2024Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modalitycited by 14
- 2023On the Local Cache Update Rules in Streaming Federated Learningcited by 8
- 2025FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learningcited by 8
- 2023Online Vertical Federated Learning for Cooperative Spectrum Sensingcited by 6
- 2024Prioritizing Modalities: Flexible Importance Scheduling in Federated Multimodal Learningcited by 4
- 2025Denoising and Adaptive Online Vertical Federated Learning for Sequential Multi-Sensor Data in Industrial Internet of Thingscited by 4
- 2025ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysiscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.