mlsanityData to 14 Oct 2026

A Multi-Modal Federated Learning Framework for Remote Sensing Image Classification

arXiv 2503.10262 ↗First posted 13 Mar 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) enables the collaborative training of deep neural networks across decentralized data archives (i.e., clients) without sharing the local data of the clients. Most of the existing FL methods assume that the data distributed across all clients is associated with the same data modality. However, remote sensing (RS) images present in different clients can be associated with diverse data modalities. The joint use of the multi-modal RS data can significantly enhance classification performance. To effectively exploit decentralized and unshared multi-modal RS data, our paper introduces a novel multi-modal FL framework for RS image classification problems. The proposed framework comprises three modules: 1) multi-modal fusion (MF); 2) feature whitening (FW); and 3) mutual information maximization (MIM). The MF module employs iterative model averaging to facilitate learning without accessing multi-modal training data on clients. The FW module aims to address the limitations of training data heterogeneity by aligning data distributions across clients. The MIM module aims to model mutual information by maximizing the similarity between images from different modalities. For the experimental analyses, we focus our attention on multi-label classification and pixel-based classification tasks in RS. The results obtained using two benchmark archives show the effectiveness of the proposed framework when compared to state-of-the-art algorithms in the literature. The code of the proposed framework will be available at https://git.tu-berlin.de/rsim/multi-modal-FL.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 51 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

26 of 51 references.

References · 26

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  4. 2021Model-Contrastive Federated Learningcited by 766
  5. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  6. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 187
  7. 2022FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correctioncited by 181
  8. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 169
  9. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 166
  10. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  11. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  12. 2022Layer-wised Model Aggregation for Personalized Federated Learningcited by 96
14 more
  1. 2021BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrievalcited by 92
  2. 2021Papaya: Practical, Private, and Scalable Federated Learningcited by 90
  3. 2023Multimodal Federated Learning via Contrastive Representation Ensemblecited by 86
  4. 2019Unsupervised Domain Adaptation using Feature-Whitening and Consensus Losscited by 67
  5. 2023FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learningcited by 60
  6. 2021Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edgecited by 56
  7. 2022On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classificationcited by 17
  8. 2020Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classificationcited by 13
  9. 2022A Novel Self-Supervised Cross-Modal Image Retrieval Method In Remote Sensingcited by 10
  10. 2023Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classificationcited by 10
  11. 2023Learning Across Decentralized Multi-Modal Remote Sensing Archives with Federated Learningcited by 8
  12. 2024Free Lunch for Federated Remote Sensing Target Fine-Grained Classification: A Parameter-Efficient Frameworkcited by 6
  13. 2024Transformer-based Federated Learning for Multi-Label Remote Sensing Image Classificationcited by 5
  14. 2023Transformer-based Multi-Modal Learning for Multi Label Remote Sensing Image Classificationcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025FedRS-Bench: Realistic Federated Learning Datasets and Benchmarks in Remote Sensingcited by 3