Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
Not on the map. Why?
Abstract (arXiv)
The rise of large language models (LLMs) has transformed healthcare by offering clinical guidance, yet their direct deployment to patients poses safety risks due to limited domain expertise. To mitigate this, we propose repositioning LLMs as clinical assistants that collaborate with experienced physicians rather than interacting with patients directly. We conduct a two-stage inspiration-feedback survey to identify real-world needs in clinical workflows. Guided by this, we construct DoctorFLAN, a large-scale Chinese medical dataset comprising 92,000 Q&A instances across 22 clinical tasks and 27 specialties. To evaluate model performance in doctor-facing applications, we introduce DoctorFLAN-test (550 single-turn Q&A items) and DotaBench (74 multi-turn conversations). Experimental results with over ten popular LLMs demonstrate that DoctorFLAN notably improves the performance of open-source LLMs in medical contexts, facilitating their alignment with physician workflows and complementing existing patient-oriented models. This work contributes a valuable resource and framework for advancing doctor-centered medical LLM development
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 13 Oct 2025.
Builds on
32 of 50 references.
References · 32
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2023Qwen Technical Reportcited by 3,273
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
- 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
- 2024Yi: Open Foundation Models by 01.AIcited by 706
- 2021Deduplicating Training Data Makes Language Models Bettercited by 705
20 more
- 2023Baichuan 2: Open Large-scale Language Modelscited by 664
- 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
- 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
- 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
- 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
- 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
- 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 161
- 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
- 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
- 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
- 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 91
- 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
- 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 89
- 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 85
- 2023BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPTcited by 64
- 2023Huatuo-26M, a Large-scale Chinese Medical QA Datasetcited by 46
- 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
- 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
- 2024Dia-LLaMA: Towards Large Language Model-driven CT Report Generationcited by 32
- 2023Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysiscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
- 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
- 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
- 2024Entry-level guide to the use of large language models for medical research
- 2023From Beginner to Expert: Modeling Medical Knowledge into General LLMs
- 2025Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration
- 2025Building a Human-Verified Clinical Reasoning Dataset via a Human LLM Hybrid Pipeline for Trustworthy Medical AI
- 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?