mlsanityData to 7 Oct 2026

Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization

arXiv 2309.07430 ↗First posted 14 Sep 2023cs.CLcited by 244

Not on the map. Why?

Abstract (arXiv)

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP), their effectiveness on a diverse range of clinical summarization tasks remains unproven. In this study, we apply adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Quantitative assessments with syntactic, semantic, and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with ten physicians evaluates summary completeness, correctness, and conciseness; in a majority of cases, summaries from our best adapted LLMs are either equivalent (45%) or superior (36%) compared to summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

Key facts. Cited by 244 papers in the map (data through 7 Oct 2026). Builds on 41 of its 103 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

41 of 103 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
29 more
  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  4. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  5. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  6. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  7. 2022UL2: Unifying Language Learning Paradigmscited by 292
  8. 2023Towards Generalist Biomedical AIcited by 286
  9. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  10. 2022Can language models learn from explanations in context?cited by 248
  11. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  12. 2016Exploring the Limits of Language Modelingcited by 172
  13. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 116
  14. 2018The Best of Both Worlds: Combining Recent Advances in Neural Machine Translationcited by 101
  15. 2023Do We Still Need Clinical Language Models?cited by 97
  16. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  17. 2023An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPTcited by 57
  18. 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 54
  19. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  20. 2023Radiology-Llama2: Best-in-Class Large Language Model for Radiologycited by 42
  21. 2018Neural Abstractive Text Summarization with Sequence-to-Sequence Modelscited by 39
  22. 2022Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibrationcited by 24
  23. 2023RadAdapt: Radiology Report Summarization via Lightweight Domain Adaptation of Large Language Modelscited by 24
  24. 2022Toward expanding the scope of radiology report summarization to multiple anatomies and modalitiescited by 14
  25. 2023SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarizationcited by 13
  26. 2023Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notescited by 11
  27. 2023CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Modelscited by 8
  28. 2023Medical Question Summarization with Entity-driven Contrastive Learningcited by 5
  29. 2020Design Considerations for High Impact, Automated Echocardiogram Analysiscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 72; 2025: 108; 2026: 62.

244 papers.

Sort:

By area:

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  3. 2024ChemLLM: A Chemical Large Language Modelcited by 100
  4. 2024Merlin: A Computed Tomography Vision-Language Foundation Model and Datasetcited by 97
  5. 2024Delving into LLM-assisted writing in biomedical publications through excess vocabularycited by 75
  6. 2024MedCalc-Bench: Evaluating Large Language Models for Medical Calculationscited by 64
  7. 2025MedHELM: Holistic Evaluation of Large Language Models for Medical Taskscited by 60
  8. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  9. 2024Towards Evaluating and Building Versatile Large Language Models for Medicinecited by 38
  10. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 36
  11. 2025Medical Large Language Model Benchmarks Should Prioritize Construct Validitycited by 36
  12. 2025A Survey of Direct Preference Optimizationcited by 35
38 more
  1. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  2. 2025TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Recordscited by 33
  3. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 31
  4. 2024Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generationcited by 30
  5. 2025Generalization Bias in Large Language Model Summarization of Scientific Researchcited by 30
  6. 2025A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domainscited by 28
  7. 2024Foundation Models in Radiology: What, How, When, Why and Why Notcited by 26
  8. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  9. 2025Large Language Models are Powerful Electronic Health Record Encoderscited by 25
  10. 2024Open (Clinical) LLMs are Sensitive to Instruction Phrasingscited by 23
  11. 2024ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaborationcited by 23
  12. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Modelscited by 22
  13. 2026HealthBench Professional: Evaluating Large Language Models on Real Clinician Chatscited by 22
  14. 2024Unlocking Memorization in Large Language Models with Dynamic Soft Promptingcited by 21
  15. 2024Can Large Language Models abstract Medical Coded Language?cited by 20
  16. 2024SurGen: Text-Guided Diffusion Model for Surgical Video Generationcited by 19
  17. 2024Overview of the First Shared Task on Clinical Text Generation: RRG24 and "Discharge Me!"cited by 19
  18. 2024HANNA: Hard-constraint Neural Network for Consistent Activity Coefficient Predictioncited by 17
  19. 2024From Single to Multi: How LLMs Hallucinate in Multi-Document Summarizationcited by 17
  20. 2023Distilling Large Language Models for Matching Patients to Clinical Trialscited by 16
  21. 2025Position: Editing Large Language Models Poses Serious Safety Riskscited by 15
  22. 2025Automated Structured Radiology Report Generationcited by 15
  23. 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Studycited by 13
  24. 2024An Evaluation of Estimative Uncertainty in Large Language Modelscited by 13
  25. 2024Large Language Models in the Clinic: A Comprehensive Benchmarkcited by 12
  26. 2025Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processingcited by 12
  27. 2025Automating MD simulations for Proteins using Large language Models: NAMD-Agentcited by 12
  28. 2024Large Language Model Capabilities in Perioperative Risk Prediction and Prognosticationcited by 11
  29. 2025BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Textcited by 11
  30. 2024A dataset and benchmark for hospital course summarization with adapted large language modelscited by 10
  31. 2024Scientific Hypothesis Generation by a Large Language Model: Laboratory Validation in Breast Cancer Treatmentcited by 10
  32. 2024Panacea: A foundation model for clinical trial search, summarization, design, and recruitmentcited by 10
  33. 2024Development of a Large Language Model-based Multi-Agent Clinical Decision Support System for Korean Triage and Acuity Scale (KTAS)-Based Triage and Treatment Planning in Emergency Departmentscited by 10
  34. 2024Speculative Coreset Selection for Task-Specific Fine-tuningcited by 10
  35. 2025Privacy-Aware RAG: Secure and Isolated Knowledge Retrievalcited by 10
  36. 2024Almanac Copilot: Towards Autonomous Electronic Health Record Navigationcited by 9
  37. 2025MAP: Evaluation and Multi-Agent Enhancement of Large Language Models for Inpatient Pathwayscited by 9
  38. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9

50 most cited of 244 All 244 (JSON)