mlsanityData to 14 Oct 2026

MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities

arXiv 2608.19788 ↗First posted 20 Aug 2026eess.IV cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Trustworthy multimodal fusion in clinical settings requires handling incomplete and heterogeneous modality subsets across institutions, where privacy constraints prohibit centralized data sharing. Federated learning (FL) mitigates data-sharing constraints but suffers from client-specific missing modalities, where institutions possess incomplete multimodal subsets, degrading fusion quality and segmentation performance. While FL and weak supervision have been studied separately, their joint use with image-level labels under heterogeneous missing modalities remains unaddressed. We propose MOSAIC, the first modality-agnostic federated framework for weakly supervised binary tumor segmentation under client-specific missing modalities. We introduce a client-specific modality-alignment module that fuses available channels into a shared latent space without prior knowledge of modality identity, a spectral prototype alignment loss that reconciles cross-client distribution shift using compact non-invertible frequency-domain statistics, and a dedicated federated refinement network that denoises the resulting CAM pseudo-labels into accurate masks, breaking the accuracy ceiling of weak supervision. Experiments on three multi-institutional brain tumor benchmarks (FeTS2022, BraTS-MEN, and BraTS-SSA) demonstrate significant improvements over all image, box, and point-supervised baselines, approaching fully supervised accuracy using only image-level labels and reaching 0.84 Dice on FeTS2022. Dynamic new client addition enables previously unseen institutions to join an already-trained federation within 0.01-0.04 Dice without retraining. Code is available at https://github.com/Tarun2201/MOSAIC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

35 of 50 references.

References · 35

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2021Model-Contrastive Federated Learningcited by 766
  6. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
  7. 2021FedBN: Federated Learning on Non-IID Features via Local Batch Normalizationcited by 571
  8. 2020FDA: Fourier Domain Adaptation for Semantic Segmentationcited by 541
  9. 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 518
  10. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  11. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 331
  12. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 312
23 more
  1. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 300
  2. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 287
  3. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 247
  4. 2018A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentationcited by 172
  5. 2022Multi-class Token Transformer for Weakly Supervised Semantic Segmentationcited by 138
  6. 2018On Regularized Losses for Weakly-supervised CNN Segmentationcited by 121
  7. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 95
  8. 2021MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluationcited by 88
  9. 2021Knowledge distillation from multi-modal to mono-modal segmentation networkscited by 68
  10. 2021HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Imagescited by 64
  11. 2022MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray imagescited by 54
  12. 2023The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningiomacited by 53
  13. 2021The Federated Tumor Segmentation (FeTS) Challengecited by 50
  14. 2019Gated CRF Loss for Weakly Supervised Semantic Image Segmentationcited by 40
  15. 2023Weakly-Supervised Semantic Segmentation with Image-Level Labels: from Traditional Models to Foundation Modelscited by 19
  16. 2024FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathologycited by 17
  17. 2023Exploring and Exploiting Uncertainty for Incomplete Multi-View Classificationcited by 13
  18. 2022FedNorm: Modality-Based Normalization in Federated Learning for Multi-Modal Liver Segmentationcited by 12
  19. 2023AME-CAM: Attentive Multiple-Exit CAM for Weakly Supervised Segmentation on MRI Brain Tumorcited by 8
  20. 2024Frequency-Based Federated Domain Generalization for Polyp Segmentationcited by 7
  21. 2024FedLPPA: Learning Personalized Prompt and Aggregation for Federated Weakly-supervised Medical Image Segmentationcited by 6
  22. 2023Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregationcited by 4
  23. 2026Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentationcited by 3

Built on by

Not cited yet.