mlsanityData to 7 Oct 2026

SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization

arXiv 2306.17384 ↗First posted 30 Jun 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Medical dialogue summarization is challenging due to the unstructured nature of medical conversations, the use of medical terminology in gold summaries, and the need to identify key information across multiple symptom sets. We present a novel system for the Dialogue2Note Medical Summarization tasks in the MEDIQA 2023 Shared Task. Our approach for section-wise summarization (Task A) is a two-stage process of selecting semantically similar dialogues and using the top-k similar dialogues as in-context examples for GPT-4. For full-note summarization (Task B), we use a similar solution with k=1. We achieved 3rd place in Task A (2nd among all teams), 4th place in Task B Division Wise Summarization (2nd among all teams), 15th place in Task A Section Header Classification (9th among all teams), and 8th place among all teams in Task B. Our results highlight the effectiveness of few-shot prompting for this task, though we also identify several weaknesses of prompting-based approaches. We compare GPT-4 performance with several finetuned baselines. We find that GPT-4 summaries are more abstractive and shorter. We make our code publicly available.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

24 of 48 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  6. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  10. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  12. 2021True Few-Shot Learning with Language Modelscited by 343
12 more
  1. 2022News Summarization and Evaluation in the Era of GPT-3cited by 324
  2. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  3. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 116
  4. 2021Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarizationcited by 76
  5. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  6. 2022Prompt Consistency for Zero-Shot Task Generalizationcited by 66
  7. 2020Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structurescited by 49
  8. 2020Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learningcited by 28
  9. 2020Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generationcited by 24
  10. 2022A Survey on Medical Document Summarizationcited by 14
  11. 2022He Said, She Said: Style Transfer for Shifting the Perspective of Dialoguescited by 1
  12. 2023Generating Query Focused Summaries without Fine-tuning the Transformer-based Pre-trained Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 4; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  2. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  3. 2024Exploring Robustness in Doctor-Patient Conversation Summarization: An Analysis of Out-of-Domain SOAP Notescited by 7
  4. 2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
  5. 2023Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reportingcited by 3
  6. 2025TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notescited by 3
  7. 2023Deciphering Diagnoses: How Large Language Models Explanations Influence Clinical Decision Makingcited by 2
  8. 2024Leveraging language models for summarizing mental state examinations: A comprehensive evaluation and dataset releasecited by 2
  9. 2025FactsR: A Safer Method for Producing High Quality Healthcare Documentationcited by 1
  10. 2024The use of large language models to enhance cancer clinical trial educational materialscited by 0
  11. 2025Leveraging Large Language Models for Rare Disease Named Entity Recognitioncited by 0
  12. 2026Membership Inference Attacks for Retrieval Based In-Context Learning for Document Question Answeringcited by 0
1 more
  1. 2026Knowledge Graph-Augmented Ambient AI for Clinical Note Generationcited by 0