mlsanityData to 7 Oct 2026

Didactic knowledge or Clinical Cases? How Data Types Shape Medical Large Language Models

arXiv 2609.22161 ↗First posted 26 Aug 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Medical large language models are commonly trained on mixtures of didactic data (e.g., textbooks) and clinical data (e.g., patient records), yet how these data types differentially shape model capabilities remains unclear. We address this issue with token-matched experiments that vary the didactic-to-clinical ratio and analyze how data composition affects performance, capability profiles, and error patterns across knowledge-intensive and clinic-oriented tasks. We uncover an asymmetric transfer across task types: clinical data improves clinic-oriented tasks while remaining competitive on knowledge-intensive ones, whereas didactic data mainly improves knowledge-intensive tasks. Error analysis suggests a knowing-doing gap, where improvements in knowledge recall do not reliably generalize to clinical reasoning. We further observe that modest amounts of clinical data yield most of the gains on EHR-grounded tasks, while the optimal mixture ratio varies with the knowledge and clinical reasoning demands of downstream tasks. These findings suggest that medical LLM data curation should be application-driven, with higher proportions of clinical data preferred for reasoning-intensive use cases.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

20 of 27 references.

References · 20

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  7. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  8. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  9. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  10. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 134
  11. 2019A Question-Entailment Approach to Question Answeringcited by 133
  12. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
8 more
  1. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  2. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 85
  3. 2024EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summariescited by 33
  4. 2023MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Modelscited by 29
  5. 2023Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinesecited by 18
  6. 2024Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMscited by 7
  7. 2026GSEM: Graph-based Self-Evolving Memory for Experience Augmented Clinical Reasoningcited by 5
  8. 2026When Correct Beliefs Collapse: Epistemic Resilience of LLMs under Clinical Pressurecited by 4

Built on by

Not cited yet.