# Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

arXiv: https://arxiv.org/abs/2305.05982 · HTML: https://mlsanity.com/paper/2305.05982/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 66 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization](https://mlsanity.com/paper/2110.07356/) (2021)
- [Complementary Explanations for Effective In-Context Learning](https://mlsanity.com/paper/2211.13892/) (2022)
- [Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures](https://mlsanity.com/paper/2009.08666/) (2020)
- [Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations](https://mlsanity.com/paper/2109.12174/) (2021)
- [Ontology-Aware Clinical Abstractive Summarization](https://mlsanity.com/paper/1905.05818/) (2019)
- [Summarizing Patients Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models](https://mlsanity.com/paper/2208.08408/) (2022)

## Built on by (most cited first, 7 of 7)

- [Extrinsically-Focused Evaluation of Omissions in Medical Summarization](https://mlsanity.com/paper/2311.08303/) (2023)
- [Clinical Insights: A Comprehensive Review of Language Models in Medicine](https://mlsanity.com/paper/2408.11735/) (2024)
- [Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting](https://mlsanity.com/paper/2311.13274/) (2023)
- [Optimizing Long-Form Clinical Text Generation with Claim-Based Rewards](https://mlsanity.com/paper/2510.02338/) (2025)
- [Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Model](https://mlsanity.com/paper/2309.09357/) (2023)
- [Personalized Clinical Note Generation from Doctor-Patient Conversations](https://mlsanity.com/paper/2408.03874/) (2024)
- [AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech](https://mlsanity.com/paper/2610.14643/) (2026)

## Abstract (arXiv)

A medical provider's summary of a patient visit serves several critical purposes, including clinical decision-making, facilitating hand-offs between providers, and as a reference for the patient. An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue, despite the complexity of patient-generated language. Even minor inaccuracies in visit summaries (for example, summarizing "patient does not have a fever" when a fever is present) can be detrimental to the outcome of care for the patient. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks that are sequentially built upon. First, we identify medical entities and their affirmations within the conversation to serve as building blocks. We study dynamically constructing few-shot prompts for tasks by conditioning on relevant patient information and use GPT-3 as the backbone for our experiments. We also develop GPT-derived summarization metrics to measure performance against reference summaries quantitatively. Both our human evaluation study and metrics for medical correctness show that summaries generated using this approach are clinically accurate and outperform the baseline approach of summarizing the dialog in a zero-shot, single-prompt setting.