mlsanityData to 7 Oct 2026

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

arXiv 2311.06025 ↗First posted 10 Nov 2023cs.CLcited by 43

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Recently, the increasing demand for superior medical services has highlighted the discrepancies in the medical infrastructure. With big data, especially texts, forming the foundation of medical services, there is an exigent need for effective natural language processing (NLP) solutions tailored to the healthcare domain. Conventional approaches leveraging pre-trained models present promising results in this domain and current large language models (LLMs) offer advanced foundation for medical text processing. However, most medical LLMs are trained only with supervised fine-tuning (SFT), even though it efficiently empowers LLMs to understand and respond to medical instructions but is ineffective in learning domain knowledge and aligning with human preference. In this work, we propose ChiMed-GPT, a new benchmark LLM designed explicitly for Chinese medical domain, and undergoes a comprehensive training regime with pre-training, SFT, and RLHF. Evaluations on tasks including information extraction, question answering, and dialogue generation demonstrate ChiMed-GPT's superior performance over general domain LLMs. Furthermore, we analyze possible biases through prompting ChiMed-GPT to perform attitude scales regarding discrimination of patients, so as to contribute to further responsible development of LLMs in the medical domain. The code and model are released at https://github.com/synlp/ChiMed-GPT.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 30 of its 60 references that are in the map. First posted on arXiv 10 Nov 2023.

Builds on

30 of 60 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
18 more
  1. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  2. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  3. 2017Mixed Precision Trainingcited by 1,135
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  6. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  7. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  8. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  9. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  10. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  11. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  12. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  13. 2022Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligencecited by 70
  14. 2019ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representationscited by 35
  15. 2021ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoderscited by 22
  16. 2023Ziya2: Data-centric Learning is All LLMs Needcited by 15
  17. 2018hyperdoc2vec: Distributed Representations of Hypertext Documentscited by 12
  18. 2020Overview of the CCKS 2019 Knowledge Graph Evaluation Track: Entity, Relation, Event and QAcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 23; 2026: 5.

43 papers.

Sort:

By area:

  1. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  2. 2023Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Taskscited by 36
  3. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  4. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 34
  5. 2025FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Trainingcited by 19
  6. 2025M-IFEval: Multilingual Instruction-Following Evaluationcited by 17
  7. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  8. 2024Large Language Model Benchmarks in Medical Taskscited by 16
  9. 20243D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Modelscited by 15
  10. 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Modelscited by 15
  11. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoningcited by 14
  12. 2024BianCang: A Traditional Chinese Medicine Large Language Modelcited by 11
31 more
  1. 2025Rethinking Diverse Human Preference Learning through Principal Component Analysiscited by 10
  2. 2024AutoMIR: Effective Zero-Shot Medical Information Retrieval without Relevance Labelscited by 8
  3. 2025Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulatorscited by 8
  4. 2025Multimodal Conditioned Diffusive Time Series Forecastingcited by 8
  5. 2024Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Modelscited by 4
  6. 2024Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Modelscited by 4
  7. 2025Odysseus Navigates the Sirens' Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generationcited by 4
  8. 2025Respond to Change with Constancy: Instruction-tuning with LLM for Non-I.I.D. Network Traffic Classificationcited by 4
  9. 2024Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answeringcited by 3
  10. 2025Affective-ROPTester: Capability and Bias Analysis of LLMs in Predicting Retinopathy of Prematuritycited by 3
  11. 2024DALL-M: Context-Aware Clinical Data Augmentation with LLMscited by 2
  12. 2024CareBot: A Pioneering Full-Process Open-Source Medical Language Modelcited by 2
  13. 2025KAG-Thinker: Interactive Thinking and Deep Reasoning in LLMs via Knowledge-Augmented Generationcited by 2
  14. 2025Recurrent Visual Feature Extraction and Stereo Attentions for CT Report Generationcited by 2
  15. 2026Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptationcited by 2
  16. 2024PclGPT: A Large Language Model for Patronizing and Condescending Language Detectioncited by 1
  17. 2025Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyondcited by 1
  18. 2025Do "New Snow Tablets" Contain Snow? Large Language Models Over-Rely on Names to Identify Ingredients of Chinese Drugscited by 1
  19. 2025DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaborationcited by 1
  20. 2025Detoxification of Large Language Models through Output-layer Fusion with a Calibration Modelcited by 1
  21. 2025Computed Tomography Visual Question Answering with Cross-modal Feature Graphingcited by 1
  22. 2025DPF-CM: A Data Processing Framework with Privacy-Preserving Vector Databases for Chinese Medical LLMs Training and Deploymentcited by 1
  23. 2025Frustratingly Easy Task-aware Pruning for Large Language Modelscited by 1
  24. 2026Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinationscited by 1
  25. 2026Silent Sabotage During Fine-Tuning: Few-Shot Rationale Poisoning of Compact Medical LLMscited by 1
  26. 2026MIND: Unified Inquiry and Diagnosis RL with Criteria Grounded Clinical Supports for Psychiatric Consultationcited by 1
  27. 2024MedGo: A Chinese Medical Large Language Modelcited by 0
  28. 2025PerMedCQA: Benchmarking Large Language Models on Medical Consumer Question Answering in Persian Languagecited by 0
  29. 2025ChiMed 2.0: Advancing Chinese Medical Dataset in Facilitating Large Language Modelingcited by 0
  30. 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarkscited by 0
  31. 2026WPG-MoE: Weak-Prior-Guided Dense Mixture-of-Experts for User-Level Social Media Depression Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model
  2. 2025Generalization of Medical Large Language Models through Cross-Domain Weak Supervision
  3. 2023ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation
  4. 2025IIMedGPT: Promoting Large Language Model Capabilities of Medical Tasks by Efficient Human Preference Alignment
  5. 2026MedLLM: An Open Medical Language Model at the Sub-Billion Scale
  6. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
  7. 2023MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models
  8. 2023Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification