# Give me Some Hard Questions: Synthetic Data Generation for Clinical QA

arXiv: https://arxiv.org/abs/2412.04573 · HTML: https://mlsanity.com/paper/2412.04573/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 6 of its 39 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Medical synthetic data and agents](https://mlsanity.com/synth/medical-synthetic-data-and-agents/)

## Builds on (most cited first, 6 of 6)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization](https://mlsanity.com/paper/2310.16427/) (2023)
- [Schema-Driven Information Extraction from Heterogeneous Tables](https://mlsanity.com/paper/2305.14336/) (2023)
- [Synthetic Data Applications in Finance](https://mlsanity.com/paper/2401.00081/) (2023)
- [Question-Answering System Extracts Information on Injection Drug Use from Clinical Notes](https://mlsanity.com/paper/2305.08777/) (2023)

## Built on by (most cited first, 5 of 5)

- [MedScore: Generalizable Factuality Evaluation of Free-Form Medical Answers by Domain-adapted Claim Decomposition and Verification](https://mlsanity.com/paper/2505.18452/) (2025)
- [Enhancing the Medical Context-Awareness Ability of LLMs via Multifaceted Self-Refinement Learning](https://mlsanity.com/paper/2511.10067/) (2025)
- [Diagnosing our datasets: How does my language model learn clinical information?](https://mlsanity.com/paper/2505.15024/) (2025)
- [RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG](https://mlsanity.com/paper/2511.04502/) (2025)
- [D-SCoRE: Document-Centric Segmentation and CoT Reasoning with Structured Export for QA-CoT Data Generation](https://mlsanity.com/paper/2508.01309/) (2025)

## Abstract (arXiv)

Clinical Question Answering (QA) systems enable doctors to quickly access patient information from electronic health records (EHRs). However, training these systems requires significant annotated data, which is limited due to the expertise needed and the privacy concerns associated with clinical data. This paper explores generating Clinical QA data using large language models (LLMs) in a zero-shot setting. We find that naive prompting often results in easy questions that do not reflect the complexity of clinical scenarios. To address this, we propose two prompting strategies: 1) instructing the model to generate questions that do not overlap with the input context, and 2) summarizing the input record using a predefined schema to scaffold question generation. Experiments on two Clinical QA datasets demonstrate that our method generates more challenging questions, significantly improving fine-tuning performance over baselines. We compare synthetic and gold data and find a gap between their training efficacy resulting from the quality of synthetically generated answers.