mlsanityData to 7 Oct 2026

Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extraction

arXiv 2204.10360 ↗First posted 21 Apr 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Relation extraction is a core problem for natural language processing in the biomedical domain. Recent research on relation extraction showed that prompt-based learning improves the performance on both fine-tuning on full training set and few-shot training. However, less effort has been made on domain-specific tasks where good prompt design can be even harder. In this paper, we investigate prompting for biomedical relation extraction, with experiments on the ChemProt dataset. We present a simple yet effective method to systematically generate comprehensive prompts that reformulate the relation extraction task as a cloze-test task under a simple prompt formulation. In particular, we experiment with different ranking scores for prompt selection. With BioMed-RoBERTa-base, our results show that prompting-based fine-tuning obtains gains by 14.21 F1 over its regular fine-tuning baseline, and 1.14 F1 over SciFive-Large, the current state-of-the-art on ChemProt. Besides, we find prompt-based learning requires fewer training examples to make reasonable predictions. The results demonstrate the potential of our methods in such a domain-specific relation extraction task.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

31 of 40 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
19 more
  1. 2019Publicly Available Clinical BERT Embeddingscited by 995
  2. 2019How Can We Know What Language Models Know?cited by 953
  3. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  4. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  5. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  6. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  7. 2020Self-Alignment Pretraining for Biomedical Entity Representationscited by 233
  8. 2018A Simple Method for Commonsense Reasoningcited by 223
  9. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  10. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  11. 2021SciFive: a text-to-text transformer model for biomedical literaturecited by 99
  12. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  13. 2020CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characterscited by 87
  14. 2021Can Language Models be Biomedical Knowledge Bases?cited by 83
  15. 2020BioMegatron: Larger Biomedical Domain Language Modelcited by 76
  16. 2020UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesauruscited by 53
  17. 2021Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learningcited by 34
  18. 2021On Training Instance Selection for Few-Shot Neural Text Generationcited by 25
  19. 2021The SelectGen Challenge: Finding the Best Training Samples for Few-Shot Neural Text Generationcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 2; 2025: 2.

5 papers.

  1. 2022Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?cited by 13
  2. 2024Prompt engineering paradigms for medical applications: scoping review and recommendations for better practicescited by 13
  3. 2024LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Languagecited by 12
  4. 2025Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language modelscited by 1
  5. 2025Error-Aware Curriculum Learning for Biomedical Relation Classificationcited by 0