# Vision-Core Guided Contrastive Learning for Balanced Multi-modal Prognosis Prediction of Stroke

arXiv: https://arxiv.org/abs/2605.14710 · HTML: https://mlsanity.com/paper/2605.14710/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 33 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 4 of 4)

- [SparseMM: Head Sparsity Emerges from Visual Concept Responses in MLLMs](https://mlsanity.com/paper/2506.05344/) (2025)
- [Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment](https://mlsanity.com/paper/2404.12634/) (2024)
- [Prediction of Thrombectomy Functional Outcomes using Multimodal Data](https://mlsanity.com/paper/2005.13061/) (2020)
- [Automatic Prediction of Stroke Treatment Outcomes: Latest Advances and Perspectives](https://mlsanity.com/paper/2412.04812/) (2024)

## Abstract (arXiv)

Deep learning and multi-modal fusion have demonstrated transformative potential in medical diagnosis by integrating diverse data sources. However, accurate prognosis for ischemic stroke remains challenging due to limitations in existing multi-modal approaches. First, current methods are predominantly confined to dual-modal fusion, lacking a framework that effectively integrates the trifecta of medical images, structured clinical data, and unstructured text. Second, they often fail to establish deep bidirectional interactions between modalities; To address these critical gaps, this paper proposes a novel tri-modal fusion model for ischemic stroke prognosis. Our approach first enriches the data representation by employing a Large Language Model (LLM) to automatically generate semi-structured diagnostic text from brain MRIs. This process not only addresses the scarcity of expert annotations but also serves as a regularized semantic enhancement, improving multimodal fusion robustness. Furthermore, we design a core component termed the Vision-Conditioned Dual Alignment Fusion Module (VDAFM), which strategically uses visual features as a conditional prior to guide fine-grained interaction with the generated text. This module achieves a dynamic and profound fusion through a dual semantic alignment loss, effectively mitigating modal heterogeneity. Extensive experiments on a real-world clinical dataset demonstrate that our model achieves state-of-the-art performance.