mlsanityData to 7 Oct 2026

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

arXiv 1906.05474 ↗First posted 13 Jun 2019cs.CLcited by 298

Not on the map. Why?

Abstract (arXiv)

Inspired by the success of the General Language Understanding Evaluation benchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to facilitate research in the development of pre-training language representations in the biomedicine domain. The benchmark consists of five tasks with ten datasets that cover both biomedical and clinical texts with different dataset sizes and difficulties. We also evaluate several baselines based on BERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and MIMIC-III clinical notes achieves the best results. We make the datasets, pre-trained models, and codes publicly available at https://github.com/ncbi-nlp/BLUE_Benchmark.

Key facts. Cited by 298 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 58; 2022: 67; 2023: 73; 2024: 51; 2025: 31; 2026: 18.

298 papers.

Sort:

By area:

  1. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  2. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
  3. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  4. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  5. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  6. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  7. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  8. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  9. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
  10. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  11. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
  12. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
38 more
  1. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  2. 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 93
  3. 2021Pre-trained Language Models in Biomedical Domain: A Systematic Surveycited by 87
  4. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  5. 2023Benchmarking large language models for biomedical natural language processing applications and recommendationscited by 78
  6. 2023LLMs Accelerate Annotation for Medical Information Extractioncited by 77
  7. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  8. 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
  9. 2023BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmarkcited by 60
  10. 2023Metacognitive Prompting Improves Understanding in Large Language Modelscited by 58
  11. 2021Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processingcited by 57
  12. 2023FactKG: Fact Verification via Reasoning on Knowledge Graphscited by 57
  13. 2021Membership Inference Attack Susceptibility of Clinical Language Modelscited by 50
  14. 2022FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processingcited by 48
  15. 2023ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Modelcited by 40
  16. 2023Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understandingcited by 40
  17. 2024SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysiscited by 40
  18. 2021Does the Magic of BERT Apply to Medical Code Assignment? A Quantitative Studycited by 39
  19. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  20. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 37
  21. 2021Improving Biomedical Pretrained Language Models with Knowledgecited by 36
  22. 2022BigBIO: A Framework for Data-Centric Biomedical Natural Language Processingcited by 36
  23. 2023A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Taskscited by 36
  24. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  25. 2023MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modelingcited by 35
  26. 2022Signature Methods in Machine Learningcited by 33
  27. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  28. 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 30
  29. 2024Knowledge Graph-Enhanced Large Language Models via Path Selectioncited by 28
  30. 2021VisualCheXbert: Addressing the Discrepancy Between Radiology Report Labels and Image Labelscited by 27
  31. 2021Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linkingcited by 27
  32. 2023DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainscited by 27
  33. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  34. 2022Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnairescited by 26
  35. 2025Clinical ModernBERT: An efficient and long context encoder for biomedical textcited by 26
  36. 2025BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLPcited by 26
  37. 2021A Survey on Extraction of Causal Relations from Natural Language Textcited by 25
  38. 2021Towards BERT-based Automatic ICD Coding: Limitations and Opportunitiescited by 25

50 most cited of 298 All 298 (JSON)