# Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data

arXiv: https://arxiv.org/abs/2402.01713 · HTML: https://mlsanity.com/paper/2402.01713/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models](https://mlsanity.com/paper/2307.02028/) (2023)
- [If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents](https://mlsanity.com/paper/2401.00812/) (2024)
- [M3Care: Learning with Missing Modalities in Multimodal Healthcare Data](https://mlsanity.com/paper/2210.17292/) (2022)
- [ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context](https://mlsanity.com/paper/1911.12216/) (2019)
- [AdaCare: Explainable Clinical Health Status Representation Learning via Scale-Adaptive Feature Extraction and Recalibration](https://mlsanity.com/paper/1911.12205/) (2019)
- [StageNet: Stage-Aware Neural Networks for Health Risk Prediction](https://mlsanity.com/paper/2001.10054/) (2020)
- [MetaPred: Meta-Learning for Clinical Risk Prediction with Limited Patient Electronic Health Records](https://mlsanity.com/paper/1905.03218/) (2019)
- [NumGPT: Improving Numeracy Ability of Generative Pre-trained Models](https://mlsanity.com/paper/2109.03137/) (2021)
- [A Comprehensive Benchmark for COVID-19 Predictive Modeling Using Electronic Health Records in Intensive Care](https://mlsanity.com/paper/2209.07805/) (2022)
- [M3Fair: Mitigating Bias in Healthcare Data through Multi-Level and Multi-Sensitive-Attribute Reweighting Method](https://mlsanity.com/paper/2306.04118/) (2023)

## Built on by (most cited first, 22 of 22)

- [Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval](https://mlsanity.com/paper/2410.04585/) (2024)
- [TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records](https://mlsanity.com/paper/2503.04176/) (2025)
- [MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks](https://mlsanity.com/paper/2505.12371/) (2025)
- [Large Language Models are Powerful Electronic Health Record Encoders](https://mlsanity.com/paper/2502.17403/) (2025)
- [REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models](https://mlsanity.com/paper/2402.07016/) (2024)
- [From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice](https://mlsanity.com/paper/2410.01812/) (2024)
- [MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling](https://mlsanity.com/paper/2508.05492/) (2025)
- [EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.21340/) (2026)
- [DeLLiriuM: A large language model for delirium prediction in the ICU using structured EHR](https://mlsanity.com/paper/2410.17363/) (2024)
- [Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints](https://mlsanity.com/paper/2604.01841/) (2026)
- [Traj-CoA: Patient Trajectory Modeling via Chain-of-Agents for Lung Cancer Risk Prediction](https://mlsanity.com/paper/2510.10454/) (2025)
- [The Power of Combining Data and Knowledge: GPT-4o is an Effective Interpreter of Machine Learning Models in Predicting Lymph Node Metastasis of Lung Cancer](https://mlsanity.com/paper/2407.17900/) (2024)
- [Predicting Lung Cancer Patient Prognosis with Large Language Models](https://mlsanity.com/paper/2408.07971/) (2024)
- [MedGellan: LLM-Generated Medical Guidance to Support Physicians](https://mlsanity.com/paper/2507.04431/) (2025)
- [OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction](https://mlsanity.com/paper/2510.17532/) (2025)
- [PaReGTA: A Temporally Aware LLM-Based Patient Representation Framework for EHR Analytics](https://mlsanity.com/paper/2602.19661/) (2026)
- [Aligning Probabilistic Beliefs under Informative Missingness: LLM Steerability in Clinical Reasoning](https://mlsanity.com/paper/2512.00479/) (2025)
- [Automatic Cardiac Risk Management Classification using large-context Electronic Patients Health Records](https://mlsanity.com/paper/2603.09685/) (2026)
- [An Underexplored Frontier: Large Language Models for Rare Disease Patient Education and Communication -- A scoping review](https://mlsanity.com/paper/2604.14179/) (2026)
- [Learning to Select Source-Traceable Evidence for Language-Model Prediction from Irregular Clinical Time Series](https://mlsanity.com/paper/2605.20292/) (2026)
- [EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events](https://mlsanity.com/paper/2609.34007/) (2026)
- [GLoC-EHR: Evidence-Cited Clinical Reasoning over Global Context and Local EHR Events](https://mlsanity.com/paper/2610.01076/) (2026)

## Abstract (arXiv)

The inherent complexity of structured longitudinal Electronic Health Records (EHR) data poses a significant challenge when integrated with Large Language Models (LLMs), which are traditionally tailored for natural language processing. Motivated by the urgent need for swift decision-making during new disease outbreaks, where traditional predictive models often fail due to a lack of historical data, this research investigates the adaptability of LLMs, like GPT-4, to EHR data. We particularly focus on their zero-shot capabilities, which enable them to make predictions in scenarios in which they haven't been explicitly trained. In response to the longitudinal, sparse, and knowledge-infused nature of EHR data, our prompting approach involves taking into account specific EHR characteristics such as units and reference ranges, and employing an in-context learning strategy that aligns with clinical contexts. Our comprehensive experiments on the MIMIC-IV and TJH datasets demonstrate that with our elaborately designed prompting framework, LLMs can improve prediction performance in key tasks such as mortality, length-of-stay, and 30-day readmission by about 35%, surpassing ML models in few-shot settings. Our research underscores the potential of LLMs in enhancing clinical decision-making, especially in urgent healthcare situations like the outbreak of emerging diseases with no labeled data. The code is publicly available at https://github.com/yhzhu99/llm4healthcare for reproducibility.