mlsanityData to 7 Oct 2026

Does Synthetic Data Generation of LLMs Help Clinical Text Mining?

arXiv 2303.04360 ↗First posted 8 Mar 2023cs.CL cs.AI cs.LGcited by 151

On the map: Synthetic data and self-improvement for LLMs → Medical synthetic data and agents

Abstract (arXiv)

Recent advancements in large language models (LLMs) have led to the development of highly potent models like OpenAI's ChatGPT. These models have exhibited exceptional performance in a variety of tasks, such as question answering, essay composition, and code generation. However, their effectiveness in the healthcare sector remains uncertain. In this study, we seek to investigate the potential of ChatGPT to aid in clinical text mining by examining its ability to extract structured information from unstructured healthcare texts, with a focus on biological named entity recognition and relation extraction. However, our preliminary results indicate that employing ChatGPT directly for these tasks resulted in poor performance and raised privacy concerns associated with uploading patients' information to the ChatGPT API. To overcome these limitations, we propose a new training paradigm that involves generating a vast quantity of high-quality synthetic data with labels utilizing ChatGPT and fine-tuning a local model for the downstream task. Our method has resulted in significant improvements in the performance of downstream tasks, improving the F1-score from 23.37% to 63.99% for the named entity recognition task and from 75.86% to 83.59% for the relation extraction task. Furthermore, generating data using ChatGPT can significantly reduce the time and effort required for data collection and labeling, as well as mitigate data privacy concerns. In summary, the proposed framework presents a promising solution to enhance the applicability of LLM models to clinical text mining.

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 17 of its 49 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

17 of 49 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  11. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  12. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
5 more
  1. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  2. 2020Exploring Chemical Space using Natural Language Processing Methodologies for Drug Discoverycited by 24
  3. 2020Med7: a transferable clinical natural language processing model for electronic health recordscited by 15
  4. 2020Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relationscited by 14
  5. 2019Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approachescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 43; 2024: 63; 2025: 31; 2026: 14.

151 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  3. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  4. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  5. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  6. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  7. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  8. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  9. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  10. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  11. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 128
  12. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
38 more
  1. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
  2. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  3. 2025Self-Adapting Language Modelscited by 71
  4. 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
  5. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  6. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 68
  7. 2023Contrasting Linguistic Patterns in Human and LLM-Generated News Textcited by 62
  8. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  9. 2023A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challengecited by 55
  10. 2023Zero-shot Temporal Relation Extraction with ChatGPTcited by 52
  11. 2024Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspectivecited by 51
  12. 2023Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languagescited by 50
  13. 2023An Empirical Study on Information Extraction using Large Language Modelscited by 46
  14. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  15. 2024Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactionscited by 43
  16. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  17. 2024Better Synthetic Data by Retrieving and Transforming Existing Datasetscited by 36
  18. 2024Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenizationcited by 32
  19. 2023Teaching Language Models to Hallucinate Less with Synthetic Taskscited by 31
  20. 2023DialogBench: Evaluating LLMs as Human-like Dialogue Systemscited by 31
  21. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 31
  22. 2023Large Language Models for Healthcare Data Augmentation: An Example on Patient-Trial Matchingcited by 30
  23. 2024Efficacy of Synthetic Data as a Benchmarkcited by 29
  24. 2023Integration of Large Language Models and Federated Learningcited by 28
  25. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 28
  26. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  27. 2023nach0: Multimodal Natural and Chemical Languages Foundation Modelcited by 26
  28. 2023Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Datacited by 26
  29. 2023Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferencescited by 25
  30. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  31. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Datacited by 21
  32. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  33. 2024Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancycited by 20
  34. 2024Aloe: A Family of Fine-tuned Open Healthcare LLMscited by 20
  35. 2023MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluationcited by 19
  36. 2024MedSyn: LLM-based Synthetic Medical Text Generation Frameworkcited by 19
  37. 2024Can LLMs Convert Graphs to Text-Attributed Graphs?cited by 19
  38. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18

50 most cited of 151 All 151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification
  2. 2025Leveraging Open-Source Large Language Models for Clinical Information Extraction in Resource-Constrained Settings
  3. 2024Large Language Models Struggle in Token-Level Clinical Named Entity Recognition
  4. 2026Fidelity, Diversity, and Privacy: A Multi-Dimensional LLM Evaluation for Clinical Data Augmentation
  5. 2025Scaling Arabic Medical Chatbots Using Synthetic Data: Enhancing Generative AI with Synthetic Patient Records
  6. 2025Benchmarking Open-Source Large Language Models on Healthcare Text Classification Tasks
  7. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation
  8. 2024LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction