# Multimodal Online Federated Learning with Modality Missing in Internet of Things

arXiv: https://arxiv.org/abs/2505.16138 · HTML: https://mlsanity.com/paper/2505.16138/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 20 of 20)

- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Federated Learning from Pre-Trained Models: A Contrastive Learning Approach](https://mlsanity.com/paper/2209.10083/) (2022)
- [Online Deep Learning: Learning Deep Neural Networks on the Fly](https://mlsanity.com/paper/1711.03705/) (2017)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning](https://mlsanity.com/paper/2308.12305/) (2023)
- [FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update](https://mlsanity.com/paper/2312.05770/) (2023)
- [Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2404.16456/) (2024)
- [Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2304.10756/) (2023)
- [Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality](https://mlsanity.com/paper/2401.13898/) (2024)
- [Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast](https://mlsanity.com/paper/2312.13508/) (2023)
- [Tighter Regret Analysis and Optimization of Online Federated Learning](https://mlsanity.com/paper/2205.06491/) (2022)
- [Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality](https://mlsanity.com/paper/2406.11048/) (2024)
- [On the Local Cache Update Rules in Streaming Federated Learning](https://mlsanity.com/paper/2303.16340/) (2023)
- [FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning](https://mlsanity.com/paper/2501.05496/) (2025)
- [Online Vertical Federated Learning for Cooperative Spectrum Sensing](https://mlsanity.com/paper/2312.11363/) (2023)
- [Prioritizing Modalities: Flexible Importance Scheduling in Federated Multimodal Learning](https://mlsanity.com/paper/2408.06549/) (2024)
- [Denoising and Adaptive Online Vertical Federated Learning for Sequential Multi-Sensor Data in Industrial Internet of Things](https://mlsanity.com/paper/2501.01693/) (2025)
- [ClusMFL: A Cluster-Enhanced Framework for Modality-Incomplete Multimodal Federated Learning in Brain Imaging Analysis](https://mlsanity.com/paper/2502.12180/) (2025)

## Built on by (most cited first, 2 of 2)

- [Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning](https://mlsanity.com/paper/2508.11159/) (2025)
- [Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detection](https://mlsanity.com/paper/2605.23984/) (2026)

## Abstract (arXiv)

The Internet of Things (IoT) ecosystem generates vast amounts of multimodal data from heterogeneous sources such as sensors, cameras, and microphones. As edge intelligence continues to evolve, IoT devices have progressed from simple data collection units to nodes capable of executing complex computational tasks. This evolution necessitates the adoption of distributed learning strategies to effectively handle multimodal data in an IoT environment. Furthermore, the real-time nature of data collection and limited local storage on edge devices in IoT call for an online learning paradigm. To address these challenges, we introduce the concept of Multimodal Online Federated Learning (MMO-FL), a novel framework designed for dynamic and decentralized multimodal learning in IoT environments. Building on this framework, we further account for the inherent instability of edge devices, which frequently results in missing modalities during the learning process. We conduct a comprehensive theoretical analysis under both complete and missing modality scenarios, providing insights into the performance degradation caused by missing modalities. To mitigate the impact of modality missing, we propose the Prototypical Modality Mitigation (PMM) algorithm, which leverages prototype learning to effectively compensate for missing modalities. Experimental results on two multimodal datasets further demonstrate the superior performance of PMM compared to benchmarks.