# Leveraging LLMs for Predicting Unknown Diagnoses from Clinical Notes

arXiv: https://arxiv.org/abs/2503.22092 · HTML: https://mlsanity.com/paper/2503.22092/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 28 Mar 2025.

## Builds on (most cited first, 12 of 12)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://mlsanity.com/paper/2203.13224/) (2022)
- [LLM-TOPLA: Efficient LLM Ensemble by Maximising Diversity](https://mlsanity.com/paper/2410.03953/) (2024)
- [Conformal Nucleus Sampling](https://mlsanity.com/paper/2305.02633/) (2023)
- [DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patients](https://mlsanity.com/paper/2309.12625/) (2023)
- [Distilling Large Language Models for Biomedical Knowledge Extraction: A Case Study on Adverse Drug Events](https://mlsanity.com/paper/2307.06439/) (2023)
- [Performance Comparison of Large Language Models on VNHSGE English Dataset: OpenAI ChatGPT, Microsoft Bing Chat, and Google Bard](https://mlsanity.com/paper/2307.02288/) (2023)
- [CBOWRA: A Representation Learning Approach for Medication Anomaly Detection](https://mlsanity.com/paper/1908.07147/) (2019)
- [Predicting Discharge Medications at Admission Time Based on Deep Learning](https://mlsanity.com/paper/1711.01386/) (2017)
- [Convolutional Neural Networks for Medical Diagnosis from Admission Notes](https://mlsanity.com/paper/1712.02768/) (2017)
- [Knowledge Base Completion for Constructing Problem-Oriented Medical Records](https://mlsanity.com/paper/2004.12905/) (2020)

## Abstract (arXiv)

Electronic Health Records (EHRs) often lack explicit links between medications and diagnoses, making clinical decision-making and research more difficult. Even when links exist, diagnosis lists may be incomplete, especially during early patient visits. Discharge summaries tend to provide more complete information, which can help infer accurate diagnoses, especially with the help of large language models (LLMs). This study investigates whether LLMs can predict implicitly mentioned diagnoses from clinical notes and link them to corresponding medications. We address two research questions: (1) Does majority voting across diverse LLM configurations outperform the best single configuration in diagnosis prediction? (2) How sensitive is majority voting accuracy to LLM hyperparameters such as temperature, top-p, and summary length? To evaluate, we created a new dataset of 240 expert-annotated medication-diagnosis pairs from 20 MIMIC-IV notes. Using GPT-3.5 Turbo, we ran 18 prompting configurations across short and long summary lengths, generating 8568 test cases. Results show that majority voting achieved 75 percent accuracy, outperforming the best single configuration at 66 percent. No single hyperparameter setting dominated, but combining deterministic, balanced, and exploratory strategies improved performance. Shorter summaries generally led to higher accuracy.In conclusion, ensemble-style majority voting with diverse LLM configurations improves diagnosis prediction in EHRs and offers a promising method to link medications and diagnoses in clinical texts.