mlsanityData to 14 Oct 2026

Towards Robust Multimodal Representation: A Unified Approach with Adaptive Experts and Alignment

arXiv 2503.09498 ↗First posted 12 Mar 2025cs.LG cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Healthcare relies on multiple types of data, such as medical images, genetic information, and clinical records, to improve diagnosis and treatment. However, missing data is a common challenge due to privacy restrictions, cost, and technical issues, making many existing multi-modal models unreliable. To address this, we propose a new multi-model model called Mixture of Experts, Symmetric Aligning, and Reconstruction (MoSARe), a deep learning framework that handles incomplete multimodal data while maintaining high accuracy. MoSARe integrates expert selection, cross-modal attention, and contrastive learning to improve feature representation and decision-making. Our results show that MoSARe outperforms existing models in situations when the data is complete. Furthermore, it provides reliable predictions even when some data are missing. This makes it especially useful in real-world healthcare settings, including resource-limited environments. Our code is publicly available at https://github.com/NazaninMn/MoSARe.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

19 of 39 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  6. 2017Self-Normalizing Neural Networkscited by 598
  7. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  8. 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
  9. 2023Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Imagescited by 134
  10. 2020Handling Missing Data with Graph Representation Learningcited by 133
  11. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  12. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
7 more
  1. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  2. 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 72
  3. 2024Transcriptomics-guided Slide Representation Learning in Computational Pathologycited by 51
  4. 2022The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillationcited by 44
  5. 2017VIGAN: Missing View Imputation with Generative Adversarial Networkscited by 33
  6. 2024ProtoGMM: Multi-prototype Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentationcited by 5
  7. 2024GenGMM: Generalized Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Structures Meet Semantics: Multimodal Fusion via Graph Contrastive Learningcited by 1
  2. 2026A Clinical Point Cloud Paradigm for In-Hospital Mortality Prediction from Multi-Level Incomplete Multimodal EHRscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024M4oE: A Foundation Model for Medical Multimodal Image Segmentation with Mixture of Experts
  2. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts
  3. 2024Multimodal Alignment and Fusion: A Survey
  4. 2022Mixture-of-experts VAEs can disregard variation in surjective multimodal data
  5. 2024Weakly-Supervised Multimodal Learning on MIMIC-CXR
  6. 2024Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications
  7. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
  8. 2026LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts