mlsanityData to 14 Oct 2026

Federated Pseudo Modality Generation for Incomplete Multi-Modal MRI Reconstruction

arXiv 2308.10910 ↗First posted 20 Aug 2023eess.IV cs.AI cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

While multi-modal learning has been widely used for MRI reconstruction, it relies on paired multi-modal data which is difficult to acquire in real clinical scenarios. Especially in the federated setting, the common situation is that several medical institutions only have single-modal data, termed the modality missing issue. Therefore, it is infeasible to deploy a standard federated learning framework in such conditions. In this paper, we propose a novel communication-efficient federated learning framework, namely Fed-PMG, to address the missing modality challenge in federated multi-modal MRI reconstruction. Specifically, we utilize a pseudo modality generation mechanism to recover the missing modality for each single-modal client by sharing the distribution information of the amplitude spectrum in frequency space. However, the step of sharing the original amplitude spectrum leads to heavy communication costs. To reduce the communication cost, we introduce a clustering scheme to project the set of amplitude spectrum into finite cluster centroids, and share them among the clients. With such an elaborate design, our approach can effectively complete the missing modality within an acceptable communication cost. Extensive experiments demonstrate that our proposed method can attain similar performance with the ideal scenario, i.e., all clients have the full set of modalities. The source code will be released.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 33 of its 62 references that are in the map. First posted on arXiv 20 Aug 2023.

Builds on

33 of 62 references.

References · 33

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  5. 2020The Future of Digital Health with Federated Learningcited by 633
  6. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  7. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  8. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
  9. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  10. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 342
  11. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 287
  12. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
21 more
  1. 2020End-to-End Variational Networks for Accelerated MRI Reconstructioncited by 202
  2. 2019Privacy-preserving Federated Brain Tumour Segmentationcited by 169
  3. 2016HeMIS: Hetero-Modal Image Segmentationcited by 103
  4. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  5. 2020Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesiscited by 86
  6. 2017Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Losscited by 82
  7. 2019Deep Learning Methods for Parallel Magnetic Resonance Image Reconstructioncited by 81
  8. 2021Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learningcited by 78
  9. 2021Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformerscited by 77
  10. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 68
  11. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 64
  12. 2022Enhancing Pseudo Label Quality for Semi-Supervised Domain-Generalized Medical Image Segmentationcited by 51
  13. 2021Multi-Modal Transformer for Accelerated MR Imagingcited by 43
  14. 2021Federated Semi-supervised Medical Image Classification via Inter-client Relation Matchingcited by 42
  15. 2021Specificity-Preserving Federated Learning for MR Image Reconstructioncited by 35
  16. 2022Federated Learning of Generative Image Priors for MRI Reconstructioncited by 33
  17. 2021Multimodal Federated Learning on IoT Datacited by 32
  18. 2018A Deep Information Sharing Network for Multi-contrast Compressed Sensing MRI Reconstructioncited by 23
  19. 2022One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesiscited by 22
  20. 2021Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Networkcited by 16
  21. 2015Fast Multi-class Dictionaries Learning with Geometrical Directions in MRI Reconstructioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 2.

5 papers.

  1. 2023Multimodal Federated Learning in Healthcare: a Reviewcited by 13
  2. 2024CAR-MFL: Cross-Modal Augmentation by Retrieval for Multimodal Federated Learning with Missing Modalitiescited by 5
  3. 2025Federated Learning for Large Models in Medical Imaging: A Comprehensive Reviewcited by 2
  4. 2026Purify-then-Align: Towards Robust Human Sensing under Modality Missing with Knowledge Distillation from Noisy Multimodal Teachercited by 0
  5. 2026FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learningcited by 0