mlsanityData to 14 Oct 2026

Prototype-guided Bilateral Alignment Multimodal Federated Learning

arXiv 2609.38925 ↗First posted 30 Sep 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal federated learning (MFL) has emerged as a pivotal paradigm for leveraging distributed data to enhance model performance. However, existing methods predominantly rely on idealized assumptions of model homogeneity and balanced modality distributions, rendering them ill-suited for practical scenarios characterized by heterogeneous client architectures and severe modality imbalance. To address these challenges, we propose a Multimodal Federated learning Prototype-guided Bilateral Alignment (MFedPBA) framework. MFedPBA facilitates robust knowledge synergy through a dual alignment mechanism: (i) at the feature level, it aligns heterogeneous feature spaces via a projection encoder optimized by contrastive learning and the Gromov-Wasserstein distance; (ii) at the decision level, it employs an entropy-weighted aggregation of naturally aligned logit prototypes. This novel design achieves robust MFL by jointly tackling heterogeneous feature spaces and collectively aggregating decisions. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art baselines under conditions of model heterogeneity and modality imbalance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 50 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

22 of 50 references.

References · 22

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  2. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
  3. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  4. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
  5. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
  6. 2020The Unbalanced Gromov Wasserstein Distance: Conic Formulation and Relaxationcited by 56
  7. 2023FedGH: Heterogeneous Federated Learning with Generalized Global Headercited by 56
  8. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  9. 2024FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learningcited by 47
  10. 2023Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Datacited by 35
  11. 2021Multimodal Federated Learning on IoT Datacited by 32
  12. 2023AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Drivingcited by 32
10 more
  1. 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
  2. 2024Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modalitycited by 14
  3. 2023Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombinationcited by 13
  4. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  5. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  6. 2023Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregationcited by 8
  7. 2025HtFLlib: A Comprehensive Heterogeneous Federated Learning Library and Benchmarkcited by 7
  8. 2024FedPIA -- Permuting and Integrating Adapters leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learningcited by 6
  9. 2025FedMobile: Enabling Knowledge Contribution-aware Multi-modal Federated Learning with Incomplete Modalitiescited by 3
  10. 2025FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Driftcited by 2

Built on by

Not cited yet.