# Is Clinical Text Enough? A Multimodal Study on Mortality Prediction in Heart Failure Patients

arXiv: https://arxiv.org/abs/2604.01924 · HTML: https://mlsanity.com/paper/2604.01924/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 10 of 10)

- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [A Thorough Examination of Decoding Methods in the Era of LLMs](https://mlsanity.com/paper/2402.06925/) (2024)
- [Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine](https://mlsanity.com/paper/2308.06834/) (2023)
- [A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction](https://mlsanity.com/paper/2208.10240/) (2022)
- [Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases](https://mlsanity.com/paper/2503.04691/) (2025)
- [Multimodal Fusion of EHR in Structures and Semantics: Integrating Clinical Records and Notes with Hypergraph and LLM](https://mlsanity.com/paper/2403.08818/) (2024)
- [Towards unstructured mortality prediction with free-text clinical notes](https://mlsanity.com/paper/1911.08437/) (2019)
- [Unlocking Anticipatory Text Generation: A Constrained Approach for Large Language Models Decoding](https://mlsanity.com/paper/2312.06149/) (2023)
- [Assessing mortality prediction through different representation models based on concepts extracted from clinical notes](https://mlsanity.com/paper/2207.10872/) (2022)

## Abstract (arXiv)

Accurate short-term mortality prediction in heart failure (HF) remains challenging, particularly when relying on structured electronic health record (EHR) data alone. We evaluate transformer-based models on a French HF cohort, comparing text-only, structured-only, multimodal, and LLM-based approaches. Our results show that enriching clinical text with entity-level representations improves prediction over CLS embeddings alone, and that supervised multimodal fusion of text and structured variables achieves the best overall performance. In contrast, large language models perform inconsistently across modalities and decoding strategies, with text-only prompts outperforming structured or multimodal inputs. These findings highlight that entity-aware multimodal transformers offer the most reliable solution for short-term HF outcome prediction, while current LLM prompting remains limited for clinical decision support.