mlsanityData to 7 Oct 2026

FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology

arXiv 2402.15858 ↗First posted 24 Feb 2024cs.CV cs.DCcited by 17

Not on the map. Why?

Abstract (arXiv)

The fusion of complementary multimodal information is crucial in computational pathology for accurate diagnostics. However, existing multimodal learning approaches necessitate access to users' raw data, posing substantial privacy risks. While Federated Learning (FL) serves as a privacy-preserving alternative, it falls short in addressing the challenges posed by heterogeneous (yet possibly overlapped) modalities data across various hospitals. To bridge this gap, we propose a Federated Multi-Modal (FedMM) learning framework that federatedly trains multiple single-modal feature extractors to enhance subsequent classification performance instead of existing FL that aims to train a unified multimodal fusion model. Any participating hospital, even with small-scale datasets or limited devices, can leverage these federated trained extractors to perform local downstream tasks (e.g., classification) while ensuring data privacy. Through comprehensive evaluations of two publicly available datasets, we demonstrate that FedMM notably outperforms two baselines in accuracy and AUC metrics.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 18 of its 36 references that are in the map. First posted on arXiv 24 Feb 2024.

Builds on

18 of 36 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  5. 2017Self-Normalizing Neural Networkscited by 591
  6. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  7. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 565
  8. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  9. 2021Federated Learning for Smart Healthcare: A Surveycited by 182
  10. 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
  11. 2020Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Resultscited by 122
  12. 2021Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learningcited by 111
6 more
  1. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 85
  2. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  3. 2020Federated Learning for Computational Pathology on Gigapixel Whole Slide Imagescited by 54
  4. 2018A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labelingcited by 37
  5. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  6. 2022CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classificationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 7.

17 papers.

Sort:

  1. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 89
  2. 2024LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinementcited by 37
  3. 2024FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularizationcited by 16
  4. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  5. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  6. 2025Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Datacited by 7
  7. 2026FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuningcited by 2
  8. 2025FSSUAVL: A Discriminative Framework using Vision Models for Federated Self-Supervised Audio and Image Understandingcited by 1
  9. 2026Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneitycited by 1
  10. 2025Adaptive Prototype Knowledge Transfer for Federated Learning with Mixed Modalities and Heterogeneous Taskscited by 0
  11. 2025Multimodal Federated Learning: A Survey through the Lens of Different FL Paradigmscited by 0
  12. 2025Role-Aware Multi-modal federated learning system for detecting phishing webpagescited by 0
5 more
  1. 2025Latency-aware Multimodal Federated Learning over UAV Networkscited by 0
  2. 2026Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learningcited by 0
  3. 2026Multimodal Graph Negative Learningcited by 0
  4. 2026Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approachcited by 0
  5. 2026MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalitiescited by 0