# Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning

arXiv: https://arxiv.org/abs/2508.11159 · HTML: https://mlsanity.com/paper/2508.11159/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [FedProto: Federated Prototype Learning across Heterogeneous Clients](https://mlsanity.com/paper/2105.00243/) (2021)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Federated Learning from Pre-Trained Models: A Contrastive Learning Approach](https://mlsanity.com/paper/2209.10083/) (2022)
- [PMR: Prototypical Modal Rebalance for Multimodal Learning](https://mlsanity.com/paper/2211.07089/) (2022)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [Deep Partial Multi-View Learning](https://mlsanity.com/paper/2011.06170/) (2020)
- [FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning](https://mlsanity.com/paper/2308.12305/) (2023)
- [Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing](https://mlsanity.com/paper/2007.09236/) (2020)
- [Multimodal Fusion on Low-quality Data: A Comprehensive Survey](https://mlsanity.com/paper/2404.18947/) (2024)
- [Diagnosing and Re-learning for Balanced Multimodal Learning](https://mlsanity.com/paper/2407.09705/) (2024)
- [Missing Modality Robustness in Semi-Supervised Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2304.10756/) (2023)
- [Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality](https://mlsanity.com/paper/2401.13898/) (2024)
- [Accelerating Hybrid Federated Learning Convergence under Partial Participation](https://mlsanity.com/paper/2304.05397/) (2023)
- [Tighter Regret Analysis and Optimization of Online Federated Learning](https://mlsanity.com/paper/2205.06491/) (2022)
- [On the Local Cache Update Rules in Streaming Federated Learning](https://mlsanity.com/paper/2303.16340/) (2023)
- [A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret](https://mlsanity.com/paper/2301.08869/) (2023)
- [Online Vertical Federated Learning for Cooperative Spectrum Sensing](https://mlsanity.com/paper/2312.11363/) (2023)
- [Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selection](https://mlsanity.com/paper/2401.00403/) (2023)
- [Prioritizing Modalities: Flexible Importance Scheduling in Federated Multimodal Learning](https://mlsanity.com/paper/2408.06549/) (2024)
- [Denoising and Adaptive Online Vertical Federated Learning for Sequential Multi-Sensor Data in Industrial Internet of Things](https://mlsanity.com/paper/2501.01693/) (2025)
- [FedHAP: Federated Hashing with Global Prototypes for Cross-silo Retrieval](https://mlsanity.com/paper/2207.05525/) (2022)
- [Multimodal Online Federated Learning with Modality Missing in Internet of Things](https://mlsanity.com/paper/2505.16138/) (2025)
- [Balanced Multi-modal Federated Learning via Cross-Modal Infiltration](https://mlsanity.com/paper/2401.00894/) (2023)

## Built on by (most cited first, 2 of 2)

- [AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2603.10043/) (2026)
- [Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detection](https://mlsanity.com/paper/2605.23984/) (2026)

## Abstract (arXiv)

The Internet of Things (IoT) ecosystem produces massive volumes of multimodal data from diverse sources, including sensors, cameras, and microphones. With advances in edge intelligence, IoT devices have evolved from simple data acquisition units into computationally capable nodes, enabling localized processing of heterogeneous multimodal data. This evolution necessitates distributed learning paradigms that can efficiently handle such data. Furthermore, the continuous nature of data generation and the limited storage capacity of edge devices demand an online learning framework. Multimodal Online Federated Learning (MMO-FL) has emerged as a promising approach to meet these requirements. However, MMO-FL faces new challenges due to the inherent instability of IoT devices, which often results in modality quantity and quality imbalance (QQI) during data collection. In this work, we systematically investigate the impact of QQI within the MMO-FL framework and present a comprehensive theoretical analysis quantifying how both types of imbalance degrade learning performance. To address these challenges, we propose the Modality Quantity and Quality Rebalanced (QQR) algorithm, a prototype learning based method designed to operate in parallel with the training process. Extensive experiments on two real-world multimodal datasets show that the proposed QQR algorithm consistently outperforms benchmarks under modality imbalance conditions with promising learning performance.