# Improving Hospital Risk Prediction with Knowledge-Augmented Multimodal EHR Modeling

arXiv: https://arxiv.org/abs/2508.01970 · HTML: https://mlsanity.com/paper/2508.01970/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 23 of 23)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [Learning to Diagnose with LSTM Recurrent Neural Networks](https://mlsanity.com/paper/1511.03677/) (2015)
- [Med42-v2: A Suite of Clinical LLMs](https://mlsanity.com/paper/2408.06142/) (2024)
- [EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records](https://mlsanity.com/paper/2401.07128/) (2024)
- [Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis](https://mlsanity.com/paper/2406.08627/) (2024)
- [GraphCare: Enhancing Healthcare Predictions with Personalized Knowledge Graphs](https://mlsanity.com/paper/2305.12788/) (2023)
- [Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval](https://mlsanity.com/paper/2410.04585/) (2024)
- [Biomedical knowledge graph-optimized prompt generation for large language models](https://mlsanity.com/paper/2311.17330/) (2023)
- [Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networks](https://mlsanity.com/paper/1807.07425/) (2018)
- [RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Records](https://mlsanity.com/paper/2403.00815/) (2024)
- [An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notes](https://mlsanity.com/paper/1802.02311/) (2018)
- [LLMs-based Few-Shot Disease Predictions using EHR: A Novel Approach Combining Predictive Agent Reasoning and Critical Agent Instruction](https://mlsanity.com/paper/2403.15464/) (2024)
- [Multimodal Fusion of EHR in Structures and Semantics: Integrating Clinical Records and Notes with Hypergraph and LLM](https://mlsanity.com/paper/2403.08818/) (2024)
- [SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction](https://mlsanity.com/paper/2405.09039/) (2024)
- [MEDFuse: Multimodal EHR Data Fusion with Masked Lab-Test Modeling and Large Language Models](https://mlsanity.com/paper/2407.12309/) (2024)
- [Predicting Chronic Disease Hospitalizations from Electronic Health Records: An Interpretable Classification Approach](https://mlsanity.com/paper/1801.01204/) (2018)
- [A Bayesian Approach to Modelling Longitudinal Data in Electronic Health Records](https://mlsanity.com/paper/1912.09086/) (2019)

## Built on by (most cited first, 2 of 2)

- [GraphDC: A Divide-and-Conquer Multi-Agent System for Scalable Graph Algorithm Reasoning](https://mlsanity.com/paper/2605.06671/) (2026)
- [BEACON-SP: Ontology-Grounded GraphRAG Framework for Clinical Suicide Risk Assessment](https://mlsanity.com/paper/2610.09026/) (2026)

## Abstract (arXiv)

Accurate prediction of clinical outcomes using Electronic Health Records (EHRs) is critical for early intervention, efficient resource allocation, and improved patient care. EHRs contain multimodal data, including both structured data and unstructured clinical notes that provide rich, context-specific information. In this work, we introduce a unified framework that seamlessly integrates these diverse modalities, leveraging all relevant available information through a two-stage architecture for clinical risk prediction. In the first stage, a fine-tuned Large Language Model (LLM) extracts crucial, task-relevant information from clinical notes, which is enhanced by graph-based retrieval of external domain knowledge from sources such as a medical corpus like PubMed, grounding the LLM's understanding. The second stage combines both unstructured representations and features derived from the structured data to generate the final predictions. This approach supports a wide range of clinical tasks. Here, we demonstrate its effectiveness on 30-day readmission and in-hospital mortality prediction. Experimental results show that our framework achieves strong performance, with AUC scores of 0.84 and 0.92, respectively, despite these tasks involving severely imbalanced datasets, with positive rates ranging from approximately 4% to 13%. Moreover, it outperforms all existing baselines and clinical practices, including established risk scoring systems. To the best of our knowledge, this is one of the first frameworks for healthcare prediction which enhances the power of an LLM-based graph-guided knowledge retrieval method by combining it with structured data for improved clinical outcome prediction.