mlsanityData to 7 Oct 2026

Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks

arXiv 2510.11040 ↗First posted 13 Oct 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The rise of large language models (LLMs) has transformed healthcare by offering clinical guidance, yet their direct deployment to patients poses safety risks due to limited domain expertise. To mitigate this, we propose repositioning LLMs as clinical assistants that collaborate with experienced physicians rather than interacting with patients directly. We conduct a two-stage inspiration-feedback survey to identify real-world needs in clinical workflows. Guided by this, we construct DoctorFLAN, a large-scale Chinese medical dataset comprising 92,000 Q&A instances across 22 clinical tasks and 27 specialties. To evaluate model performance in doctor-facing applications, we introduce DoctorFLAN-test (550 single-turn Q&A items) and DotaBench (74 multi-turn conversations). Experimental results with over ten popular LLMs demonstrate that DoctorFLAN notably improves the performance of open-source LLMs in medical contexts, facilitating their alignment with physician workflows and complementing existing patient-oriented models. This work contributes a valuable resource and framework for advancing doctor-centered medical LLM development

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

32 of 50 references.

References · 32

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  9. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  10. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  11. 2024Yi: Open Foundation Models by 01.AIcited by 706
  12. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
20 more
  1. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  2. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  3. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  4. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  5. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  6. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  7. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 161
  8. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  9. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
  10. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
  11. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 91
  12. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  13. 2023DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultationcited by 89
  14. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluationcited by 85
  15. 2023BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPTcited by 64
  16. 2023Huatuo-26M, a Large-scale Chinese Medical QA Datasetcited by 46
  17. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
  18. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  19. 2024Dia-LLaMA: Towards Large Language Model-driven CT Report Generationcited by 32
  20. 2023Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysiscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
  2. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  3. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
  4. 2024Entry-level guide to the use of large language models for medical research
  5. 2023From Beginner to Expert: Modeling Medical Knowledge into General LLMs
  6. 2025Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration
  7. 2025Building a Human-Verified Clinical Reasoning Dataset via a Human LLM Hybrid Pipeline for Trustworthy Medical AI
  8. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?