mlsanityData to 7 Oct 2026

LLMs in Biomedicine: A study on clinical Named Entity Recognition

arXiv 2404.07376 ↗First posted 10 Apr 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable versatility in various NLP tasks but encounter distinct challenges in biomedical due to the complexities of language and data scarcity. This paper investigates LLMs application in the biomedical domain by exploring strategies to enhance their performance for the NER task. Our study reveals the importance of meticulously designed prompts in the biomedical. Strategic selection of in-context examples yields a marked improvement, offering ~15-20% increase in F1 score across all benchmark datasets for biomedical few-shot NER. Additionally, our results indicate that integrating external biomedical knowledge via prompting strategies can enhance the proficiency of general-purpose LLMs to meet the specialized needs of biomedical NER. Leveraging a medical knowledge base, our proposed method, DiRAG, inspired by Retrieval-Augmented Generation (RAG), can boost the zero-shot F1 score of LLMs for biomedical NER. Code is released at urlhttps://github.com/masoud-monajati/LLMBioNER

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 62 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

36 of 62 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  10. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  11. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  12. 2019Publicly Available Clinical BERT Embeddingscited by 995
24 more
  1. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  2. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  3. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  4. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  5. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  6. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  7. 2023InstructUIE: Multi-task Instruction Tuning for Unified Information Extractioncited by 167
  8. 2023What Makes Good Examples for Visual In-Context Learning?cited by 150
  9. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  10. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  11. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  12. 2015Yara Parser: A Fast and Accurate Dependency Parsercited by 110
  13. 2023Do We Still Need Clinical Language Models?cited by 97
  14. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  15. 2021DEGREE: A Data-Efficient Generation-Based Event Extraction Modelcited by 82
  16. 2023What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?cited by 47
  17. 2021GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domaincited by 36
  18. 2023GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Promptscited by 34
  19. 2020HunFlair: An Easy-to-Use Tool for State-of-the-Art Biomedical Named Entity Recognitioncited by 26
  20. 2022DICE: Data-Efficient Clinical Event Extraction with Generative Modelscited by 20
  21. 2023Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processingcited by 16
  22. 2023Zero-shot information extraction from radiological reports using ChatGPTcited by 15
  23. 2023MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answeringcited by 12
  24. 2024Retrieval-Augmented Data Augmentation for Low-Resource Domain Taskscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 4.

12 papers.

  1. 2024Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcarecited by 12
  2. 2024RAMIE: Retrieval-Augmented Multi-task Information Extraction with Large Language Models on Dietary Supplementscited by 11
  3. 2024Multi-Document Financial Question Answering using LLMscited by 6
  4. 2025Do LLMs Surpass Encoders for Biomedical NER?cited by 6
  5. 2025Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Languagecited by 3
  6. 2026Leveraging Large Language Models and Survival Analysis for Early Prediction of Chemotherapy Outcomescited by 1
  7. 2026FMI@SU ToxHabits: Evaluating LLMs Performance on Toxic Habit Extraction in Spanish Clinical Textscited by 1
  8. 2025Enhancing LLMs for Identifying and Prioritizing Important Medical Jargons from Electronic Health Record Notes Utilizing Data Augmentation: A Comparative Studycited by 0
  9. 2025LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognitioncited by 0
  10. 2025Harnessing Large Language Models for Biomedical Named Entity Recognitioncited by 0
  11. 2026A Mechanism and Optimization Study on the Impact of Information Density on User-Generated Content Named Entity Recognitioncited by 0
  12. 2026Implicit Framing in Obstetric Counseling Notes: A Grounded LLM Pipeline on a VBAC-Eligible Cohortcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction
  2. 2025Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
  3. 2024Large Language Models Struggle in Token-Level Clinical Named Entity Recognition
  4. 2023Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition
  5. 2024A Survey for Large Language Models in Biomedicine
  6. 2024A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences
  7. 2024Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data
  8. 2024LLM-IE: A Python Package for Generative Information Extraction with Large Language Models