The Potential of LLMs in Medical Education: Generating Questions and Answers for Qualification Exams
Not on the map. Why?
Abstract (arXiv)
In this work, we leverage LLMs to produce medical qualification exam questions and the corresponding answers through few-shot prompts, investigating in-depth how LLMs meet the requirements in terms of coherence, evidence of statement, factual consistency, and professionalism etc. Utilizing a multicenter bidirectional anonymized database with respect to comorbid chronic diseases, named Elderly Comorbidity Medical Database (CECMed), we tasked LLMs with generating open-ended questions and answers based on a subset of sampled admission reports. For CECMed, the retrospective cohort includes patients enrolled from January 2010 to January 2022 while the prospective cohort from January 2023 to November 2023, with participants sourced from selected tertiary and community hospitals across the southern, northern, and central regions of China. A total of 8 widely used LLMs were used, including ERNIE 4, ChatGLM 4, Doubao, Hunyuan, Spark 4, Qwen, Conventional medical education requires sophisticated clinicians to formulate questions and answers based on prototypes from EHRs, which is heuristic and time-consuming. We found that mainstream LLMs could generate questions and answers with real-world EHRs at levels close to clinicians. Although current LLMs performed dissatisfactory in some aspects, medical students, interns and residents could reasonably make use of LLMs to facilitate understanding.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 31 Oct 2024.
Builds on
17 of 30 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
- 2023How Language Model Hallucinations Can Snowballcited by 311
- 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
- 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 162
- 2020MedDialog: Two Large-scale Medical Dialogue Datasetscited by 161
- 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 131
5 more
- 2024BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Textcited by 86
- 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
- 2024Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approachescited by 52
- 2021Semi-Supervised Variational Reasoning for Medical Dialogue Generationcited by 30
- 2024Closing the gap between open-source and commercial large language models for medical evidence summarizationcited by 21
Built on by
Citing papers per year: 2026: 2.
2 papers.