mlsanityData to 7 Oct 2026

PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain

arXiv 2310.14151 ↗First posted 22 Oct 2023cs.CLcited by 40

Not on the map. Why?

Abstract (arXiv)

Biomedical language understanding benchmarks are the driving forces for artificial intelligence applications with large language model (LLM) back-ends. However, most current benchmarks: (a) are limited to English which makes it challenging to replicate many of the successes in English for other languages, or (b) focus on knowledge probing of LLMs and neglect to evaluate how LLMs apply these knowledge to perform on a wide range of bio-medical tasks, or (c) have become a publicly available corpus and are leaked to LLMs during pre-training. To facilitate the research in medical LLMs, we re-build the Chinese Biomedical Language Understanding Evaluation (CBLUE) benchmark into a large scale prompt-tuning benchmark, PromptCBLUE. Our benchmark is a suitable test-bed and an online platform for evaluating Chinese LLMs' multi-task capabilities on a wide range bio-medical tasks including medical entity recognition, medical text classification, medical natural language inference, medical dialogue understanding and medical content/dialogue generation. To establish evaluation on these tasks, we have experimented and report the results with the current 9 Chinese LLMs fine-tuned with differtent fine-tuning techniques.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 44 of its 68 references that are in the map. First posted on arXiv 22 Oct 2023.

Builds on

44 of 68 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  12. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
32 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  9. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  11. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  12. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  13. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  14. 2021GPT Understands, Toocited by 715
  15. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  16. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  17. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  18. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  19. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  20. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  21. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  22. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  23. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 99
  24. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  25. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  26. 2022A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasetscited by 45
  27. 2022SciNLI: A Corpus for Natural Language Inference on Scientific Textcited by 27
  28. 2023F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Taskscited by 18
  29. 2022DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processingcited by 14
  30. 2023Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translationcited by 13
  31. 2020AutoRC: Improving BERT Based Relation Classification Models via Architecture Searchcited by 9
  32. 2021Building Chinese Biomedical Language Models via Multi-Level Text Discriminationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 18; 2025: 11; 2026: 8.

40 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  2. 2023Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Taskscited by 36
  3. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  4. 2025LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validationcited by 29
  5. 2024MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuningcited by 19
  6. 2024TCMBench: A Comprehensive Benchmark for Evaluating Large Language Models in Traditional Chinese Medicinecited by 16
  7. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  8. 2024Large Language Model Benchmarks in Medical Taskscited by 16
  9. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  10. 2023From Beginner to Expert: Modeling Medical Knowledge into General LLMscited by 14
  11. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  12. 2024MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokenscited by 13
28 more
  1. 2025InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Trainingcited by 11
  2. 2025PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustmentcited by 9
  3. 2024PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdificationcited by 8
  4. 2024MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Expertscited by 7
  5. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
  6. 2024Text2MDT: Extracting Medical Decision Trees from Medical Textscited by 6
  7. 2025AECBench: A Hierarchical Benchmark for Knowledge Evaluation of Large Language Models in the AEC Fieldcited by 6
  8. 2024Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decodingcited by 5
  9. 2025TCM-Ladder: A Benchmark for Multimodal Question Answering on Traditional Chinese Medicinecited by 5
  10. 2024MedCare: Advancing Medical LLMs through Decoupling Clinical Alignment and Knowledge Aggregationcited by 4
  11. 2026MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicinecited by 4
  12. 2024HALO: Hallucination Analysis and Learning Optimization to Empower LLMs with Retrieval-Augmented Context for Guided Clinical Decision Makingcited by 3
  13. 20243DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selectioncited by 3
  14. 2026ShapLoRA: Allocation of Low-rank Adaption on Large Language Models via Shapley Value Inspired Importance Estimationcited by 3
  15. 2024CPSDBench: A Large Language Model Evaluation Benchmark and Baseline for Chinese Public Security Domaincited by 2
  16. 2024A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithmcited by 1
  17. 2024IAPT: Instruction-Aware Prompt Tuning for Large Language Modelscited by 1
  18. 2025OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmologycited by 1
  19. 2025Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyondcited by 1
  20. 2025DynaSwarm: Dynamically Graph Structure Selection for LLM-based Multi-agent Systemcited by 1
  21. 2026Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyondcited by 1
  22. 2026Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domaincited by 1
  23. 2026ACL: Aligned Contrastive Learning Improves BERT and Multi-exit BERT Fine-tuningcited by 1
  24. 2026MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reportscited by 1
  25. 2024Advancing Chinese biomedical text mining with community challengescited by 0
  26. 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarkscited by 0
  27. 2026A Generative Model for Joint Multiple Intent Detection and Slot Fillingcited by 0
  28. 2026MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultationcited by 0