mlsanityData to 14 Oct 2026

Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue

arXiv 2308.03549 ↗First posted 7 Aug 2023cs.CLcited by 119

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have achieved remarkable breakthroughs in understanding and responding to user intents. However, their performance lag behind general use cases in some expertise domains, such as Chinese medicine. Existing efforts to incorporate Chinese medicine into LLMs rely on Supervised Fine-Tuning (SFT) with single-turn and distilled dialogue data. These models lack the ability for doctor-like proactive inquiry and multi-turn comprehension and cannot align responses with experts' intentions. In this work, we introduce Zhongjing, the first Chinese medical LLaMA-based LLM that implements an entire training pipeline from continuous pre-training, SFT, to Reinforcement Learning from Human Feedback (RLHF). Additionally, we construct a Chinese multi-turn medical dialogue dataset of 70,000 authentic doctor-patient dialogues, CMtMedQA, which significantly enhances the model's capability for complex dialogue and proactive inquiry initiation. We also define a refined annotation rule and evaluation criteria given the unique characteristics of the biomedical domain. Extensive experimental results show that Zhongjing outperforms baselines in various capacities and matches the performance of ChatGPT in some abilities, despite the 100x parameters. Ablation studies also demonstrate the contributions of each component: pre-training enhances medical knowledge, and RLHF further improves instruction-following ability and safety. Our code, datasets, and models are available at https://github.com/SupritYoung/Zhongjing.

Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 30 of its 43 references that are in the map. First posted on arXiv 7 Aug 2023.

Builds on

30 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023HuatuoGPT, towards Taming Language Model to Be a Doctor24 May

Other references · 29

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  11. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  12. 2023LIMA: Less Is More for Alignmentcited by 1,100
17 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  2. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  3. 2023Instruction Tuning with GPT-4cited by 686
  4. 2023Textbooks Are All You Needcited by 569
  5. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  6. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  7. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  8. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  9. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 305
  10. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  11. 2023The False Promise of Imitating Proprietary LLMscited by 263
  12. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  13. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  14. 2023HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledgecited by 162
  15. 2023DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Taskcited by 131
  16. 2022Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligencecited by 70
  17. 2023Huatuo-26M, a Large-scale Chinese Medical QA Datasetcited by 46

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 45; 2025: 47; 2026: 17.

119 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  2. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  4. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
  5. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  6. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  7. 2023HuatuoGPT-II, One-stage Training for Medical Adaption of LLMscited by 92
  8. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  9. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  10. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  11. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  12. 2024Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactionscited by 46
38 more
  1. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  2. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 44
  3. 2025Improving Medical Large Vision-Language Models with Abnormal-Aware Feedbackcited by 42
  4. 2024FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agentscited by 40
  5. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  6. 2023Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Taskscited by 36
  7. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  8. 2024TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systemscited by 28
  9. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  10. 2023HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responsescited by 25
  11. 2023Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Modelcited by 24
  12. 2024A Survey of Conversational Searchcited by 22
  13. 2023User Modeling in the Era of Large Language Models: Current Research and Future Directionscited by 18
  14. 2024SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Taskscited by 18
  15. 2024Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultationcited by 17
  16. 2024Large Language Model Benchmarks in Medical Taskscited by 17
  17. 2024PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applicationscited by 16
  18. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  19. 2024MedAide: Information Fusion and Anatomy of Medical Intents via LLM-based Agent Collaborationcited by 16
  20. 2025Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Supportcited by 16
  21. 20253MDBench: Medical Multimodal Multi-agent Dialogue Benchmarkcited by 16
  22. 2025Multimodal Large Language Models for Medicine: A Comprehensive Surveycited by 16
  23. 2024Qibo: A Large Language Model for Traditional Chinese Medicinecited by 13
  24. 2024Large Language Models in the Clinic: A Comprehensive Benchmarkcited by 13
  25. 2025Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruningcited by 13
  26. 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 12
  27. 2025PII-Bench: Evaluating Query-Aware Privacy Protection Systemscited by 11
  28. 2024FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examinationcited by 10
  29. 2025OpenTCM: A GraphRAG-Empowered LLM-based System for Traditional Chinese Medicine Knowledge Retrieval and Diagnosiscited by 10
  30. 2024MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Mediacited by 9
  31. 2025Evaluating the Sensitivity of LLMs to Prior Contextcited by 9
  32. 2024JMedBench: A Benchmark for Evaluating Japanese Biomedical Large Language Modelscited by 8
  33. 2024CTourLLM: Enhancing LLMs with Chinese Tourism Knowledgecited by 7
  34. 2025J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domaincited by 7
  35. 2025Trustworthy Medical Question Answering: An Evaluation-Centric Surveycited by 7
  36. 2025ShizhenGPT: Towards Multimodal LLMs for Traditional Chinese Medicinecited by 7
  37. 2025A Multi-Agent Framework with Automated Decision Rule Optimization for Cross-Domain Misinformation Detectioncited by 6
  38. 2025WenyanGPT: A Large Language Model for Classical Chinese Taskscited by 6

50 most cited of 119 All 119 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023IvyGPT: InteractiVe Chinese pathwaY language model in medical domain
  2. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences
  3. 2025MedBench v4: A Robust and Scalable Benchmark for Evaluating Chinese Medical Language Models, Multimodal Models, and Intelligent Agents
  4. 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
  5. 2025MTCMB: A Multi-Task Benchmark Framework for Evaluating LLMs on Knowledge, Reasoning, and Safety in Traditional Chinese Medicine
  6. 2025BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine
  7. 2023MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
  8. 2025TCM-Eval: An Expert-Level Dynamic and Extensible Benchmark for Traditional Chinese Medicine