# HuatuoGPT, towards Taming Language Model to Be a Doctor

arXiv: https://arxiv.org/abs/2305.15075 · HTML: https://mlsanity.com/paper/2305.15075/

## Key facts

- Cited by 214 papers in the map (data through 14 Oct 2026).
- Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 24 May 2023.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Medical LLM alignment](https://mlsanity.com/alignment/medical-llm-alignment/)

## Builds on (most cited first, 18 of 18)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data](https://mlsanity.com/paper/2304.08247/) (2023)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [MedDialog: Two Large-scale Medical Dialogue Datasets](https://mlsanity.com/paper/2004.03329/) (2020)
- [DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task](https://mlsanity.com/paper/2304.01097/) (2023)
- [CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2106.08087/) (2021)
- [End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis](https://mlsanity.com/paper/1901.10623/) (2019)
- [Huatuo-26M, a Large-scale Chinese Medical QA Dataset](https://mlsanity.com/paper/2305.01526/) (2023)
- [Phoenix: Democratizing ChatGPT across Languages](https://mlsanity.com/paper/2304.10453/) (2023)
- [PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model](https://mlsanity.com/paper/2203.17090/) (2022)

## Built on by (most cited first, 30 of 214)

- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [A Survey on Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2402.13116/) (2024)
- [Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model](https://mlsanity.com/paper/2311.13231/) (2023)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale](https://mlsanity.com/paper/2406.19280/) (2024)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue](https://mlsanity.com/paper/2308.03549/) (2023)
- [CMB: A Comprehensive Medical Benchmark in Chinese](https://mlsanity.com/paper/2308.08833/) (2023)
- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [HuatuoGPT-II, One-stage Training for Medical Adaption of LLMs](https://mlsanity.com/paper/2311.09774/) (2023)
- [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://mlsanity.com/paper/2308.14346/) (2023)
- [Lifelong Learning of Large Language Model based Agents: A Roadmap](https://mlsanity.com/paper/2501.07278/) (2025)
- [EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education](https://mlsanity.com/paper/2308.02773/) (2023)
- [Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions](https://mlsanity.com/paper/2404.03264/) (2024)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [AceGPT, Localizing Large Language Models in Arabic](https://mlsanity.com/paper/2309.12053/) (2023)
- [Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory](https://mlsanity.com/paper/2311.08719/) (2023)
- [Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning](https://mlsanity.com/paper/2508.16949/) (2025)
- [LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models](https://mlsanity.com/paper/2306.12420/) (2023)
- ["It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents](https://mlsanity.com/paper/2309.11653/) (2023)
- [Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application](https://mlsanity.com/paper/2407.01885/) (2024)
- [A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future](https://mlsanity.com/paper/2504.12328/) (2025)
- [A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine](https://mlsanity.com/paper/2405.08603/) (2024)
- [ChatCounselor: A Large Language Models for Mental Health Support](https://mlsanity.com/paper/2309.15461/) (2023)
- [CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis](https://mlsanity.com/paper/2407.13301/) (2024)
- [Baichuan-M1: Pushing the Medical Capability of Large Language Models](https://mlsanity.com/paper/2502.12671/) (2025)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)

## Abstract (arXiv)

In this paper, we present HuatuoGPT, a large language model (LLM) for medical consultation. The core recipe of HuatuoGPT is to leverage both textitdistilled data from ChatGPT and textitreal-world data from doctors in the supervised fine-tuned stage. The responses of ChatGPT are usually detailed, well-presented and informative while it cannot perform like a doctor in many aspects, e.g. for integrative diagnosis. We argue that real-world data from doctors would be complementary to distilled data in the sense the former could tame a distilled language model to perform like doctors. To better leverage the strengths of both data, we train a reward model to align the language model with the merits that both data bring, following an RLAIF (reinforced learning from AI feedback) fashion. To evaluate and benchmark the models, we propose a comprehensive evaluation scheme (including automatic and manual metrics). Experimental results demonstrate that HuatuoGPT achieves state-of-the-art results in performing medical consultation among open-source LLMs in GPT-4 evaluation, human evaluation, and medical benchmark datasets. It is worth noting that by using additional real-world data and RLAIF, the distilled language model (i.e., HuatuoGPT) outperforms its teacher model ChatGPT in most cases. Our code, data, and models are publicly available at urlhttps://github.com/FreedomIntelligence/HuatuoGPT. The online demo is available at urlhttps://www.HuatuoGPT.cn/.