mlsanityData to 14 Oct 2026

Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization

arXiv 2110.07356 ↗First posted 9 Sep 2021cs.CL cs.AI cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

In medical dialogue summarization, summaries must be coherent and must capture all the medically relevant information in the dialogue. However, learning effective models for summarization require large amounts of labeled data which is especially hard to obtain. We present an algorithm to create synthetic training data with an explicit focus on capturing medically relevant information. We utilize GPT-3 as the backbone of our algorithm and scale 210 human labeled examples to yield results comparable to using 6400 human labeled examples (~30x) leveraging low-shot learning and an ensemble method. In detailed experiments, we show that this approach produces high quality training data that can further be combined with human labeled data to get summaries that are strongly preferable to those produced by models trained on human data alone both in terms of medical accuracy and coherency.

Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 9 Sep 2021.

Builds on

19 of 28 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  6. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  7. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  8. 2020Recipes for building an open-domain chatbotcited by 593
  9. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  10. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 246
  11. 2020ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-trainingcited by 222
  12. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 218
7 more
  1. 2018Learning to Summarize Radiology Findingscited by 54
  2. 2020Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structurescited by 49
  3. 2018Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Actscited by 44
  4. 2019Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversationscited by 41
  5. 2018Extractive Summarization of EHR Discharge Notescited by 12
  6. 2020Extracting Structured Data from Physician-Patient Conversations By Predicting Noteworthy Utterancescited by 8
  7. 2019Clinical Concept Extraction: a Methodology Reviewcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 11; 2023: 23; 2024: 28; 2025: 7; 2026: 3.

76 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  2. 2023A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4cited by 81
  3. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  4. 2023NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Modelscited by 59
  5. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  6. 2022Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generationcited by 33
  7. 2022PriMock57: A Dataset Of Primary Care Mock Consultationscited by 31
  8. 2024A Survey of Small Language Modelscited by 31
  9. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  10. 2023The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRscited by 29
  11. 2023Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Datacited by 26
  12. 2024MediTOD: An English Dialogue Dataset for Medical History Taking with Comprehensive Annotationscited by 22
38 more
  1. 2022Retrieval augmentation of large language models for lay language generationcited by 21
  2. 2023IC3: Image Captioning by Committee Consensuscited by 20
  3. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  4. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  5. 2024Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Educationcited by 15
  6. 2024See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknessescited by 15
  7. 2021Natural Language Processing for Smart Healthcarecited by 14
  8. 2022A Survey on Medical Document Summarizationcited by 14
  9. 2022User-Driven Research of Medical Note Generation Softwarecited by 13
  10. 2023SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarizationcited by 13
  11. 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Studycited by 13
  12. 2024Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Gamescited by 12
  13. 2024Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Datacited by 12
  14. 2022a survey on GPT-3cited by 9
  15. 2023Conversational AI-Powered Design: ChatGPT as Designer, User, and Productcited by 8
  16. 2023What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarizationcited by 8
  17. 2023Experience and Evidence are the eyes of an excellent summarizer! Towards Knowledge Infused Multi-modal Clinical Conversation Summarizationcited by 8
  18. 2023WangLab at MEDIQA-Chat 2023: Clinical Note Generation from Doctor-Patient Conversations using Large Language Modelscited by 7
  19. 2023Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language modelscited by 7
  20. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  21. 2024A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approachescited by 7
  22. 2024Improving Clinical Note Generation from Complex Doctor-Patient Conversationcited by 7
  23. 2022Pay Attention to Your Tone: Introducing a New Dataset for Polite Language Rewritecited by 5
  24. 2023Adding guardrails to advanced chatbotscited by 5
  25. 2023Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognitioncited by 5
  26. 2025Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarizationcited by 5
  27. 2022Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translationcited by 4
  28. 2024On The Persona-based Summarization of Domain-Specific Documentscited by 4
  29. 2024Game Development as Human-LLM Interactioncited by 4
  30. 2024A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunitiescited by 4
  31. 2024Toward Relieving Clinician Burden by Automatically Generating Progress Notes using Interim Hospital Datacited by 4
  32. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 4
  33. 2025Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applicationscited by 4
  34. 2021MEDCOD: A Medically-Accurate, Emotive, Diverse, and Controllable Dialog Systemcited by 3
  35. 2022Leveraging pre-trained language models for conversational information seeking from textcited by 3
  36. 2023A Survey for Biomedical Text Summarization: From Pre-trained to Large Language Modelscited by 3
  37. 2023Injecting knowledge into language generation: a case study in auto-charting after-visit care instructions from medical dialoguecited by 3
  38. 2023From Military to Healthcare: Adopting and Expanding Ethical Principles for Generative Artificial Intelligencecited by 3

50 most cited of 76 All 76 (JSON)