# DRG-LLaMA : Tuning LLaMA Model to Predict Diagnosis-related Group for Hospitalized Patients

arXiv: https://arxiv.org/abs/2309.12625 · HTML: https://mlsanity.com/paper/2309.12625/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 60 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Explainable Prediction of Medical Codes from Clinical Text](https://mlsanity.com/paper/1802.05695/) (2018)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [Metrics for Multi-Class Classification: an Overview](https://mlsanity.com/paper/2008.05756/) (2020)
- [DocBERT: BERT for Document Classification](https://mlsanity.com/paper/1904.08398/) (2019)

## Built on by (most cited first, 30 of 30)

- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [Zero Shot Health Trajectory Prediction Using Transformer](https://mlsanity.com/paper/2407.21124/) (2024)
- [Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Data](https://mlsanity.com/paper/2401.06866/) (2024)
- [A dataset and benchmark for hospital course summarization with adapted large language models](https://mlsanity.com/paper/2403.05720/) (2024)
- [Towards Adapting Open-Source Large Language Models for Expert-Level Clinical Note Generation](https://mlsanity.com/paper/2405.00715/) (2024)
- [A Perspective for Adapting Generalist AI to Specialized Medical AI Applications and Their Challenges](https://mlsanity.com/paper/2411.00024/) (2024)
- [Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction](https://mlsanity.com/paper/2501.17326/) (2025)
- [Reinforcement Learning for Out-of-Distribution Reasoning in LLMs: An Empirical Study on Diagnosis-Related Group Coding](https://mlsanity.com/paper/2505.21908/) (2025)
- [MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling](https://mlsanity.com/paper/2508.05492/) (2025)
- [Development and Testing of a Novel Large Language Model-Based Clinical Decision Support Systems for Medication Safety in 12 Clinical Specialties](https://mlsanity.com/paper/2402.01741/) (2024)
- [From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine](https://mlsanity.com/paper/2502.09242/) (2025)
- [Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications](https://mlsanity.com/paper/2509.08604/) (2025)
- [Interpretable Differential Diagnosis with Dual-Inference Large Language Models](https://mlsanity.com/paper/2407.07330/) (2024)
- [Toward Relieving Clinician Burden by Automatically Generating Progress Notes using Interim Hospital Data](https://mlsanity.com/paper/2410.12845/) (2024)
- [Entry-level guide to the use of large language models for medical research](https://mlsanity.com/paper/2410.18856/) (2024)
- [Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis](https://mlsanity.com/paper/2412.00056/) (2024)
- [Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Models](https://mlsanity.com/paper/2502.16491/) (2025)
- [Large Language Models for Healthcare Text Classification: A Systematic Review](https://mlsanity.com/paper/2503.01159/) (2025)
- [The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare](https://mlsanity.com/paper/2411.03287/) (2024)
- [Rephrasing Electronic Health Records for Pretraining Clinical Language Models](https://mlsanity.com/paper/2411.18940/) (2024)
- [MedGellan: LLM-Generated Medical Guidance to Support Physicians](https://mlsanity.com/paper/2507.04431/) (2025)
- [When Metrics Disagree: Automatic Similarity vs. LLM-as-a-Judge for Clinical Dialogue Evaluation](https://mlsanity.com/paper/2603.00314/) (2026)
- [Data Augmentation Techniques for Chinese Disease Name Normalization](https://mlsanity.com/paper/2501.01195/) (2025)
- [A Multi-Modal Deep Learning Framework for Pan-Cancer Prognosis](https://mlsanity.com/paper/2501.07016/) (2025)
- [Review GIDE -- Restaurant Review Gastrointestinal Illness Detection and Extraction with Large Language Models](https://mlsanity.com/paper/2503.09743/) (2025)
- [Leveraging LLMs for Predicting Unknown Diagnoses from Clinical Notes](https://mlsanity.com/paper/2503.22092/) (2025)
- [The NordDRG AI Benchmark for Large Language Models](https://mlsanity.com/paper/2506.13790/) (2025)
- [Accelerating Reproducible Research in Synthetic EHR Generation](https://mlsanity.com/paper/2606.06990/) (2026)
- [Hidden Data Challenges in Clinical Machine Learning](https://mlsanity.com/paper/2610.13350/) (2026)

## Abstract (arXiv)

In the U.S. inpatient payment system, the Diagnosis-Related Group (DRG) is pivotal, but its assignment process is inefficient. The study introduces DRG-LLaMA, an advanced large language model (LLM) fine-tuned on clinical notes to enhance DRGs assignment. Utilizing LLaMA as the foundational model and optimizing it through Low-Rank Adaptation (LoRA) on 236,192 MIMIC-IV discharge summaries, our DRG-LLaMA-7B model exhibited a noteworthy macro-averaged F1 score of 0.327, a top-1 prediction accuracy of 52.0%, and a macro-averaged Area Under the Curve (AUC) of 0.986, with a maximum input token length of 512. This model surpassed the performance of prior leading models in DRG prediction, showing a relative improvement of 40.3% and 35.7% in macro-averaged F1 score compared to ClinicalBERT and CAML, respectively. Applied to base DRG and complication or comorbidity (CC)/major complication or comorbidity (MCC) prediction, DRG-LLaMA achieved a top-1 prediction accuracy of 67.8% and 67.5%, respectively. Additionally, our findings indicate that DRG-LLaMA's performance correlates with increased model parameters and input context lengths.