# Large Language Models to Identify Social Determinants of Health in Electronic Health Records

arXiv: https://arxiv.org/abs/2308.06354 · HTML: https://mlsanity.com/paper/2308.06354/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 68 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Do We Still Need Clinical Language Models?](https://mlsanity.com/paper/2302.08091/) (2023)
- [Launching into clinical space with medspaCy: a new clinical text processing toolkit in Python](https://mlsanity.com/paper/2106.07799/) (2021)
- [Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing](https://mlsanity.com/paper/2305.16635/) (2023)
- [The 2022 n2c2/UW Shared Task on Extracting Social Determinants of Health](https://mlsanity.com/paper/2301.05571/) (2023)
- [Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification](https://mlsanity.com/paper/2304.02496/) (2023)
- [A Study of Social and Behavioral Determinants of Health in Lung Cancer Patients Using Transformers-based Natural Language Processing Models](https://mlsanity.com/paper/2108.04949/) (2021)
- [Prompt-based Extraction of Social Determinants of Health Using Few-shot Learning](https://mlsanity.com/paper/2306.07170/) (2023)
- [Leveraging Natural Language Processing to Augment Structured Social Determinants of Health Data in the Electronic Health Record](https://mlsanity.com/paper/2212.07538/) (2022)
- [Natural language processing to automatically extract the presence and severity of esophagitis in notes of patients undergoing radiotherapy](https://mlsanity.com/paper/2303.13722/) (2023)
- [BCH-NLP at BioCreative VII Track 3: medications detection in tweets using transformer networks and multi-task learning](https://mlsanity.com/paper/2111.13726/) (2021)

## Built on by (most cited first, 30 of 55)

- [DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios](https://mlsanity.com/paper/2410.23746/) (2024)
- [EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summaries](https://mlsanity.com/paper/2402.16040/) (2024)
- [Transforming Wearable Data into Personal Health Insights using Large Language Model Agents](https://mlsanity.com/paper/2406.06464/) (2024)
- [Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education](https://mlsanity.com/paper/2409.18924/) (2024)
- [Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysis](https://mlsanity.com/paper/2403.16303/) (2024)
- [A Review on Generative AI Models for Synthetic Medical Text, Time Series, and Longitudinal Data](https://mlsanity.com/paper/2411.12274/) (2024)
- [Human-AI Co-design for Clinical Prediction Models](https://mlsanity.com/paper/2601.09072/) (2026)
- [LLMCARE: early detection of cognitive impairment via transformer models enhanced by LLM-generated synthetic data](https://mlsanity.com/paper/2508.10027/) (2025)
- [HEARTS: A Holistic Framework for Explainable, Sustainable and Robust Text Stereotype Detection](https://mlsanity.com/paper/2409.11579/) (2024)
- [Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimization](https://mlsanity.com/paper/2409.12741/) (2024)
- [Leveraging Large Language Models to Extract Information on Substance Use Disorder Severity from Clinical Notes: A Zero-shot Learning Approach](https://mlsanity.com/paper/2403.12297/) (2024)
- [Assessing and Mitigating Medical Knowledge Drift and Conflicts in Large Language Models](https://mlsanity.com/paper/2505.07968/) (2025)
- [A comparative study of zero-shot inference with large language models and supervised modeling in breast cancer pathology classification](https://mlsanity.com/paper/2401.13887/) (2024)
- [Synth-SBDH: A Synthetic Dataset of Social and Behavioral Determinants of Health for Clinical Text](https://mlsanity.com/paper/2406.06056/) (2024)
- [Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation](https://mlsanity.com/paper/2409.09135/) (2024)
- [Disease Entity Recognition and Normalization is Improved with Large Language Model Derived Synthetic Normalized Mentions](https://mlsanity.com/paper/2410.07951/) (2024)
- [Large Language Models for Healthcare Text Classification: A Systematic Review](https://mlsanity.com/paper/2503.01159/) (2025)
- [Cancer Type, Stage and Prognosis Assessment from Pathology Reports using LLMs](https://mlsanity.com/paper/2503.01194/) (2025)
- [Integration of Large Language Models and Traditional Deep Learning for Social Determinants of Health Prediction](https://mlsanity.com/paper/2505.04655/) (2025)
- [A Scoping Review of Synthetic Data Generation by Language Models in Biomedical Research and Application: Data Utility and Quality Perspectives](https://mlsanity.com/paper/2506.16594/) (2025)
- [Detecting the Clinical Features of Difficult-to-Treat Depression using Synthetic Data from Large Language Models](https://mlsanity.com/paper/2402.07645/) (2024)
- [Extracting Social Support and Social Isolation Information from Clinical Psychiatry Notes: Comparing a Rule-based NLP System and a Large Language Model](https://mlsanity.com/paper/2403.17199/) (2024)
- [Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data](https://mlsanity.com/paper/2403.19511/) (2024)
- [Leveraging Social Determinants of Health in Alzheimer's Research Using LLM-Augmented Literature Mining and Knowledge Graphs](https://mlsanity.com/paper/2410.09080/) (2024)
- [Revealing Treatment Non-Adherence Bias in Clinical Machine Learning Using Large Language Models](https://mlsanity.com/paper/2502.19625/) (2025)
- [Benchmarking Ethical and Safety Risks of Healthcare LLMs in China-Toward Systemic Governance under Healthy China 2030](https://mlsanity.com/paper/2505.07205/) (2025)
- [PyHealth 2.0: A Comprehensive Open-Source Toolkit for Accessible and Reproducible Clinical Deep Learning](https://mlsanity.com/paper/2601.16414/) (2026)
- [SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH)](https://mlsanity.com/paper/2407.17126/) (2024)
- [A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Health](https://mlsanity.com/paper/2411.02594/) (2024)
- [Detecting Children with Autism Spectrum Disorder based on Script-Centric Behavior Understanding with Emotional Enhancement](https://mlsanity.com/paper/2411.09413/) (2024)

## Abstract (arXiv)

Social determinants of health (SDoH) have an important impact on patient outcomes but are incompletely collected from the electronic health records (EHR). This study researched the ability of large language models to extract SDoH from free text in EHRs, where they are most commonly documented, and explored the role of synthetic clinical text for improving the extraction of these scarcely documented, yet extremely valuable, clinical data. 800 patient notes were annotated for SDoH categories, and several transformer-based models were evaluated. The study also experimented with synthetic data generation and assessed for algorithmic bias. Our best-performing models were fine-tuned Flan-T5 XL (macro-F1 0.71) for any SDoH, and Flan-T5 XXL (macro-F1 0.70). The benefit of augmenting fine-tuning with synthetic data varied across model architecture and size, with smaller Flan-T5 models (base and large) showing the greatest improvements in performance (delta F1 +0.12 to +0.23). Model performance was similar on the in-hospital system dataset but worse on the MIMIC-III dataset. Our best-performing fine-tuned models outperformed zero- and few-shot performance of ChatGPT-family models for both tasks. These fine-tuned models were less likely than ChatGPT to change their prediction when race/ethnicity and gender descriptors were added to the text, suggesting less algorithmic bias (p<0.05). At the patient-level, our models identified 93.8% of patients with adverse SDoH, while ICD-10 codes captured 2.0%. Our method can effectively extracted SDoH information from clinic notes, performing better compare to GPT zero- and few-shot settings. These models could enhance real-world evidence on SDoH and aid in identifying patients needing social support.