# LLMs for Doctors: Leveraging Medical LLMs to Assist Doctors, Not Replace Them

arXiv: https://arxiv.org/abs/2406.18034 · HTML: https://mlsanity.com/paper/2406.18034/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [HuatuoGPT, towards Taming Language Model to Be a Doctor](https://mlsanity.com/paper/2305.15075/) (2023)
- [CMB: A Comprehensive Medical Benchmark in Chinese](https://mlsanity.com/paper/2308.08833/) (2023)
- [CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2106.08087/) (2021)
- [HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs](https://mlsanity.com/paper/2311.09774/) (2023)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://mlsanity.com/paper/2308.14346/) (2023)
- [ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation](https://mlsanity.com/paper/2306.09968/) (2023)
- [BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPT](https://mlsanity.com/paper/2310.15896/) (2023)
- [Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques](https://mlsanity.com/paper/2005.01795/) (2020)
- [Huatuo-26M, a Large-scale Chinese Medical QA Dataset](https://mlsanity.com/paper/2305.01526/) (2023)
- [User-Driven Research of Medical Note Generation Software](https://mlsanity.com/paper/2205.02549/) (2022)
- [Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysis](https://mlsanity.com/paper/2310.11722/) (2023)

## Built on by (most cited first, 5 of 5)

- [CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis](https://mlsanity.com/paper/2407.13301/) (2024)
- [MedSyn: Enhancing Diagnostics with Human-AI Collaboration](https://mlsanity.com/paper/2506.14774/) (2025)
- [MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning Correction](https://mlsanity.com/paper/2605.08094/) (2026)
- [Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care](https://mlsanity.com/paper/2605.08533/) (2026)
- [Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds](https://mlsanity.com/paper/2605.18827/) (2026)

## Abstract (arXiv)

The recent success of Large Language Models (LLMs) has had a significant impact on the healthcare field, providing patients with medical advice, diagnostic information, and more. However, due to a lack of professional medical knowledge, patients are easily misled by generated erroneous information from LLMs, which may result in serious medical problems. To address this issue, we focus on tuning the LLMs to be medical assistants who collaborate with more experienced doctors. We first conduct a two-stage survey by inspiration-feedback to gain a broad understanding of the real needs of doctors for medical assistants. Based on this, we construct a Chinese medical dataset called DoctorFLAN to support the entire workflow of doctors, which includes 92K Q&A samples from 22 tasks and 27 specialists. Moreover, we evaluate LLMs in doctor-oriented scenarios by constructing the DoctorFLAN-textittest containing 550 single-turn Q&A and DotaBench containing 74 multi-turn conversations. The evaluation results indicate that being a medical assistant still poses challenges for existing open-source models, but DoctorFLAN can help them significantly. It demonstrates that the doctor-oriented dataset and benchmarks we construct can complement existing patient-oriented work and better promote medical LLMs research.