FedMobile: Enabling Knowledge Contribution-aware Multi-modal Federated Learning with Incomplete Modalities
Not on the map. Why?
Abstract (arXiv)
The Web of Things (WoT) enhances interoperability across web-based and ubiquitous computing platforms while complementing existing IoT standards. The multimodal Federated Learning (FL) paradigm has been introduced to enhance WoT by enabling the fusion of multi-source mobile sensing data while preserving privacy. However, a key challenge in mobile sensing systems using multimodal FL is modality incompleteness, where some modalities may be unavailable or only partially captured, potentially degrading the system's performance and reliability. Current multimodal FL frameworks typically train multiple unimodal FL subsystems or apply interpolation techniques on the node side to approximate missing modalities. However, these approaches overlook the shared latent feature space among incomplete modalities across different nodes and fail to discriminate against low-quality nodes. To address this gap, we present FedMobile, a new knowledge contribution-aware multimodal FL framework designed for robust learning despite missing modalities. FedMobile prioritizes local-to-global knowledge transfer, leveraging cross-node multimodal feature information to reconstruct missing features. It also enhances system performance and resilience to modality heterogeneity through rigorous node contribution assessments and knowledge contribution-aware aggregation rules. Empirical evaluations on five widely recognized multimodal benchmark datasets demonstrate that FedMobile maintains robust learning even when up to 90% of modality information is missing or when data from two modalities are randomly missing, outperforming state-of-the-art baselines.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 50 references that are in the map. First posted on arXiv 20 Feb 2025.
Builds on
18 of 50 references.
References · 18
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Deep Leakage from Gradientscited by 1,004
- 2018Federated Learning for Mobile Keyboard Predictioncited by 565
- 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
- 2022Algorithms to estimate Shapley value feature attributionscited by 114
- 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
- 2020Privacy-preserving Traffic Flow Prediction: A Federated Learning Approachcited by 82
- 2020Deep Anomaly Detection for Time-series Data in Industrial IoT: A Communication-Efficient On-device Federated Learning Approachcited by 56
- 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
- 2015Deep Multimodal Learning for Audio-Visual Speech Recognitioncited by 37
- 2023AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Drivingcited by 32
6 more
- 2018R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answeringcited by 23
- 2024Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modalitycited by 21
- 2021A Survey of Privacy Vulnerabilities of Mobile Device Sensorscited by 18
- 2023Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrastcited by 17
- 2020Toward Understanding the Influence of Individual Clients in Federated Learningcited by 16
- 2022FLAME: Federated Learning Across Multi-device Environmentscited by 8
Built on by
Citing papers per year: 2026: 3.
3 papers.