A Knowledge-Injection Framework for Zero-Shot Adaptation of LLMs to Delirium Prediction
Not on the map. Why?
Abstract (arXiv)
Large language models show promise for clinical prediction, but zero-shot performance on specialized tasks is limited by incomplete domain knowledge, especially for smaller locally deployable models. We present a lightweight knowledge-injection framework for zero-shot ICU delirium prediction that augments a deterministic natural-language summary of structured electronic health record data with an external clinical knowledge report at inference time, without fine-tuning or retrieval. We evaluate LLaMA 3.1 8B and LLaMA 3.3 70B on 3,160 ICU admissions from the MIMIC IV dataset. Adding a clinically meaningful external knowledge report improves AUROC by 8.57 percentage points for the 8B model and 1.99 percentage points for the 70B model compared to no external knowledge. Relative to a GPT-5.2 frontier-model reference without external knowledge report (AUROC 68.86%), knowledge injection reduces the performance gap from 15.66 to 7.09 AUROC points for LLaMA 8B and from 5.30 to 3.31 AUROC points for LLaMA 70B. Random control reports do not improve performance and often degrade it, indicating that gains depend on clinically meaningful content rather than added prompt length alone. SHAP-based attribution further confirms that the injected knowledge is actively used during prediction. These findings suggest that inference-time knowledge injection can narrow the gap between locally deployable open-weight models and frontier closed models while preserving a practical, privacy-preserving workflow for resource-constrained clinical settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 55 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
19 of 55 references.
References · 19
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
- 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 141
- 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 123
- 2025MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilotcited by 83
- 2024A Closer Look at the Limitations of Instruction Tuningcited by 77
- 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 70
- 2023Matching Patients to Clinical Trials with Large Language Modelscited by 69
7 more
- 2021Knowledge-aware Zero-Shot Learning: Survey and Perspectivecited by 41
- 2025Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoningcited by 31
- 2023A Systematic Review of Few-Shot Learning in Medical Imagingcited by 28
- 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
- 2023Large Language Models Leverage External Knowledge to Extend Clinical Insight Beyond Language Boundariescited by 10
- 2024LLMs are not Zero-Shot Reasoners for Biomedical Information Extractioncited by 9
- 2025Will Large Language Models Transform Clinical Prediction?cited by 3
Built on by
Not cited yet.