# FedMobile: Enabling Knowledge Contribution-aware Multi-modal Federated Learning with Incomplete Modalities

arXiv: https://arxiv.org/abs/2502.15839 · HTML: https://mlsanity.com/paper/2502.15839/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 18 of 18)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining](https://mlsanity.com/paper/2203.07320/) (2022)
- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [Privacy-preserving Traffic Flow Prediction: A Federated Learning Approach](https://mlsanity.com/paper/2003.08725/) (2020)
- [Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approach](https://mlsanity.com/paper/2007.09712/) (2020)
- [FedMultimodal: A Benchmark For Multimodal Federated Learning](https://mlsanity.com/paper/2306.09486/) (2023)
- [Deep Multimodal Learning for Audio-Visual Speech Recognition](https://mlsanity.com/paper/1501.05396/) (2015)
- [AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving](https://mlsanity.com/paper/2302.08646/) (2023)
- [R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering](https://mlsanity.com/paper/1805.09701/) (2018)
- [Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality](https://mlsanity.com/paper/2401.13898/) (2024)
- [A Survey of Privacy Vulnerabilities of Mobile Device Sensors](https://mlsanity.com/paper/2106.10154/) (2021)
- [Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast](https://mlsanity.com/paper/2312.13508/) (2023)
- [Toward Understanding the Influence of Individual Clients in Federated Learning](https://mlsanity.com/paper/2012.10936/) (2020)
- [FLAME: Federated Learning Across Multi-device Environments](https://mlsanity.com/paper/2202.08922/) (2022)

## Built on by (most cited first, 3 of 3)

- [Reconstructing Training Data from Adapter-based Federated Large Language Models](https://mlsanity.com/paper/2601.17533/) (2026)
- [GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems](https://mlsanity.com/paper/2603.18540/) (2026)
- [Prototype-guided Bilateral Alignment Multimodal Federated Learning](https://mlsanity.com/paper/2609.38925/) (2026)

## Abstract (arXiv)

The Web of Things (WoT) enhances interoperability across web-based and ubiquitous computing platforms while complementing existing IoT standards. The multimodal Federated Learning (FL) paradigm has been introduced to enhance WoT by enabling the fusion of multi-source mobile sensing data while preserving privacy. However, a key challenge in mobile sensing systems using multimodal FL is modality incompleteness, where some modalities may be unavailable or only partially captured, potentially degrading the system's performance and reliability. Current multimodal FL frameworks typically train multiple unimodal FL subsystems or apply interpolation techniques on the node side to approximate missing modalities. However, these approaches overlook the shared latent feature space among incomplete modalities across different nodes and fail to discriminate against low-quality nodes. To address this gap, we present FedMobile, a new knowledge contribution-aware multimodal FL framework designed for robust learning despite missing modalities. FedMobile prioritizes local-to-global knowledge transfer, leveraging cross-node multimodal feature information to reconstruct missing features. It also enhances system performance and resilience to modality heterogeneity through rigorous node contribution assessments and knowledge contribution-aware aggregation rules. Empirical evaluations on five widely recognized multimodal benchmark datasets demonstrate that FedMobile maintains robust learning even when up to 90% of modality information is missing or when data from two modalities are randomly missing, outperforming state-of-the-art baselines.