mlsanityData to 14 Oct 2026

UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?

arXiv 2306.16931 ↗First posted 29 Jun 2023cs.CL cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

This paper presents UMASS_BioNLP team participation in the MEDIQA-Chat 2023 shared task for Task-A and Task-C. We focus especially on Task-C and propose a novel LLMs cooperation system named a doctor-patient loop to generate high-quality conversation data sets. The experiment results demonstrate that our approaches yield reasonable performance as evaluated by automatic metrics such as ROUGE, medical concept recall, BLEU, and Self-BLEU. Furthermore, we conducted a comparative analysis between our proposed method and ChatGPT and GPT-4. This analysis also investigates the potential of utilizing cooperation LLMs to generate high-quality datasets.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 49 of its 80 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

49 of 80 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  8. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  11. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
37 more
  1. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  2. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  3. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  4. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  5. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
  6. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
  7. 2022Is GPT-3 a Good Data Annotator?cited by 208
  8. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
  9. 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 161
  10. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 154
  11. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 145
  12. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 140
  13. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 135
  14. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 118
  15. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 104
  16. 2021Can Language Models be Biomedical Knowledge Bases?cited by 83
  17. 2018A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literaturecited by 79
  18. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 62
  19. 2021Launching into clinical space with medspaCy: a new clinical text processing toolkit in Pythoncited by 40
  20. 2020Question-Driven Summarization of Answers to Consumer Health Questionscited by 39
  21. 2023Language Models are Few-shot Learners for Prognostic Predictioncited by 37
  22. 2022PriMock57: A Dataset Of Primary Care Mock Consultationscited by 31
  23. 2022MedJEx: A Medical Jargon Extraction Model with Wiki's Hyperlink Span and Contextualized Masked Language Model Scorecited by 20
  24. 2020MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretrainingcited by 18
  25. 2022Multi-label Few-shot ICD Coding as Autoregressive Generation with Promptcited by 17
  26. 2022Learning to Ask Like a Physiciancited by 16
  27. 2022Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Contextcited by 15
  28. 2022Context Variance Evaluation of Pretrained Language Models for Prompt-based Biomedical Knowledge Probingcited by 15
  29. 2020Zero-shot Entity Linking with Efficient Long Range Sequence Modelingcited by 13
  30. 2021Improving Formality Style Transfer with Context-Aware Rule Injectioncited by 13
  31. 2020Towards an Automated SOAP Note: Classifying Utterances from Medical Conversationscited by 10
  32. 2023An Investigation of Evaluation Metrics for Automated Medical Note Generationcited by 10
  33. 2020Extracting Structured Data from Physician-Patient Conversations By Predicting Noteworthy Utterancescited by 8
  34. 2023Causality-aware Concept Extraction based on Knowledge-guided Promptingcited by 8
  35. 2023Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyondcited by 8
  36. 2022Automated Identification of Eviction Status from Electronic Health Record Notescited by 6
  37. 2022An Automatic SOAP Classification System Using Weakly Supervision And Transfer Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 3; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  2. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  3. 2023BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processingcited by 25
  4. 2023PaniniQA: Enhancing Patient Education Through Interactive Question Answeringcited by 21
  5. 2024PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queriescited by 19
  6. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  7. 2023Improving Summarization with Human Editscited by 7
  8. 2023Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarizationcited by 5
  9. 2023EHRTutor: Enhancing Patient Understanding of Discharge Instructionscited by 4
  10. 2025Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applicationscited by 4
  11. 2025A Typology of Synthetic Datasets for Dialogue Processing in Clinical Contextscited by 1
  12. 2025Using Medical Algorithms for Task-Oriented Dialogue in LLM-Based Medical Interviewscited by 0
2 more
  1. 2026Generating High Quality Synthetic Data for Dutch Medical Conversationscited by 0
  2. 2026Clinical Communication Processing with Models Trained on LLM-Generated Synthetic Data: A Structured Survey and Novel Application Case Studiescited by 0