# The Foundational Capabilities of Large Language Models in Predicting Postoperative Risks Using Clinical Notes

arXiv: https://arxiv.org/abs/2402.17493 · HTML: https://mlsanity.com/paper/2402.17493/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 72 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://mlsanity.com/paper/2101.11038/) (2021)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models](https://mlsanity.com/paper/2302.07257/) (2023)
- [Probabilistic FastText for Multi-Sense Word Embeddings](https://mlsanity.com/paper/1806.02901/) (2018)

## Built on by (most cited first, 6 of 6)

- [HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models](https://mlsanity.com/paper/2405.07460/) (2024)
- [Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records](https://mlsanity.com/paper/2510.19014/) (2025)
- [Generative Foundation Model for Structured and Unstructured Electronic Health Records](https://mlsanity.com/paper/2508.16054/) (2025)
- [Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set](https://mlsanity.com/paper/2509.06463/) (2025)
- [Resource-Conscious Modeling for Next- Day Discharge Prediction Using Clinical Notes](https://mlsanity.com/paper/2604.03498/) (2026)
- [Clinically Grounded Privacy Evaluation of Medical LMs](https://mlsanity.com/paper/2606.09590/) (2026)

## Abstract (arXiv)

Clinical notes recorded during a patient's perioperative journey holds immense informational value. Advances in large language models (LLMs) offer opportunities for bridging this gap. Using 84,875 pre-operative notes and its associated surgical cases from 2018 to 2021, we examine the performance of LLMs in predicting six postoperative risks using various fine-tuning strategies. Pretrained LLMs outperformed traditional word embeddings by an absolute AUROC of 38.3% and AUPRC of 33.2%. Self-supervised fine-tuning further improved performance by 3.2% and 1.5%. Incorporating labels into training further increased AUROC by 1.8% and AUPRC by 2%. The highest performance was achieved with a unified foundation model, with improvements of 3.6% for AUROC and 2.6% for AUPRC compared to self-supervision, highlighting the foundational capabilities of LLMs in predicting postoperative risks, which could be potentially beneficial when deployed for perioperative care