mlsanityData to 14 Oct 2026

A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMs

arXiv 2504.14657 ↗First posted 20 Apr 2025cs.CL cs.AI cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Synthetic Electronic Health Records (EHRs) offer a valuable opportunity to create privacy preserving and harmonized structured data, supporting numerous applications in healthcare. Key benefits of synthetic data include precise control over the data schema, improved fairness and representation of patient populations, and the ability to share datasets without concerns about compromising real individuals privacy. Consequently, the AI community has increasingly turned to Large Language Models (LLMs) to generate synthetic data across various domains. However, a significant challenge in healthcare is ensuring that synthetic health records reliably generalize across different hospitals, a long standing issue in the field. In this work, we evaluate the current state of commercial LLMs for generating synthetic data and investigate multiple aspects of the generation process to identify areas where these models excel and where they fall short. Our main finding from this work is that while LLMs can reliably generate synthetic health records for smaller subsets of features, they struggle to preserve realistic distributions and correlations as the dimensionality of the data increases, ultimately limiting their ability to generalize across diverse hospital settings.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 32 of its 61 references that are in the map. First posted on arXiv 20 Apr 2025.

Builds on

32 of 61 references.

References · 32

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  4. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  5. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  6. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  7. 2023Improving Text Embeddings with Large Language Modelscited by 303
  8. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  9. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  10. 2018Why Is My Classifier Discriminatory?cited by 143
  11. 2021DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networkscited by 109
  12. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 94
20 more
  1. 2023MOTOR: A Time-To-Event Foundation Model For Structured Medical Recordscited by 78
  2. 2024MedExQA: Medical Question Answering Benchmark with Multiple Explanationscited by 60
  3. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  4. 2024Large Language Models for Disease Diagnosis: A Scoping Reviewcited by 51
  5. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  6. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesiscited by 42
  7. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 36
  8. 2024A Continued Pretrained LLM Approach for Automatic Medical Note Generationcited by 30
  9. 2025Clinical ModernBERT: An efficient and long context encoder for biomedical textcited by 26
  10. 2024Emergency Department Decision Support using Clinical Pseudo-notescited by 22
  11. 2024Schema Matching with Large Language Models: an Experimental Studycited by 20
  12. 2024MedSyn: LLM-based Synthetic Medical Text Generation Frameworkcited by 19
  13. 2024Text Serialization and Their Relationship with the Conventional Paradigms of Tabular Machine Learningcited by 17
  14. 2024Differentially Private Tabular Data Synthesis using Large Language Modelscited by 16
  15. 2023Integrating UMLS Knowledge into Large Language Models for Medical Question Answeringcited by 10
  16. 2018Generalizability of predictive models for intensive care unit patientscited by 9
  17. 2023Does Synthetic Data Make Large Language Models More Efficient?cited by 8
  18. 2024Enhancing Antibiotic Stewardship using a Natural Language Approach for Better Feature Representationcited by 8
  19. 2024FEET: A Framework for Evaluating Embedding Techniquescited by 8
  20. 2024MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Datacited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

By area:

  1. 2025HiMAE: Hierarchical Masked Autoencoders Discover Resolution-Specific Structure in Wearable Time Seriescited by 19
  2. 2025Structured Semantics from Unstructured Notes: Language Model Approaches to EHR-Based Decision Supportcited by 7
  3. 2025Serialized EHR make for good text representationscited by 7
  4. 2026Learning Clinical Representations Under Systematic Distribution Shiftcited by 2
  5. 2026Discriminative Representation Learning for Clinical Predictioncited by 2
  6. 2025Infherno: End-to-end Agent-based FHIR Resource Synthesis from Free-form Clinical Notescited by 1
  7. 2026Learning Longitudinal Health Representations from EHR and Wearable Datacited by 1
  8. 2026Uncertainty-Aware Foundation Models for Clinical Datacited by 1
  9. 2026Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguesecited by 1
  10. 2026Dense Feature Learning via Linear Structure Preservation in Medical Datacited by 0
  11. 2026Event Fields: Learning Latent Event Structure for Waveform Foundation Modelscited by 0
  12. 2026WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Recordscited by 0
1 more
  1. 2026MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveformscited by 0