mlsanityData to 7 Oct 2026

KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction

arXiv 2104.07650 ↗First posted 15 Apr 2021cs.CL cs.AI cs.IRcited by 180

Not on the map. Why?

Abstract (arXiv)

Recently, prompt-tuning has achieved promising results for specific few-shot classification tasks. The core idea of prompt-tuning is to insert text pieces (i.e., templates) into the input and transform a classification task into a masked language modeling problem. However, for relation extraction, determining an appropriate prompt template requires domain expertise, and it is cumbersome and time-consuming to obtain a suitable label word. Furthermore, there exists abundant semantic and prior knowledge among the relation labels that cannot be ignored. To this end, we focus on incorporating knowledge among relation labels into prompt-tuning for relation extraction and propose a Knowledge-aware Prompt-tuning approach with synergistic optimization (KnowPrompt). Specifically, we inject latent knowledge contained in relation labels into prompt construction with learnable virtual type words and answer words. Then, we synergistically optimize their representation with structured constraints. Extensive experimental results on five datasets with standard and low-resource settings demonstrate the effectiveness of our approach. Our code and datasets are available in https://github.com/zjunlp/KnowPrompt for reproducibility.

Key facts. Cited by 180 papers in the map (data through 7 Oct 2026). Builds on 44 of its 92 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

44 of 92 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  9. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  10. 2021GPT Understands, Toocited by 715
  11. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  12. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
32 more
  1. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  2. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  3. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  4. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  5. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  6. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  7. 2021How Many Data Points is a Prompt Worth?cited by 197
  8. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  9. 2018Graph Convolution over Pruned Dependency Trees Improves Relation Extractioncited by 146
  10. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  11. 2020Reasoning with Latent Structure Refinement for Document-Level Relation Extractioncited by 97
  12. 2020TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Taskcited by 96
  13. 2019Enriching Pre-trained Language Model with Entity Information for Relation Classificationcited by 82
  14. 2019Attention Guided Graph Convolutional Networks for Relation Extractioncited by 82
  15. 2021An Improved Baseline for Sentence-level Relation Extractioncited by 80
  16. 2020Dialogue-Based Relation Extractioncited by 68
  17. 2019SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominalscited by 63
  18. 2019OpenNRE: An Open and Extensible Toolkit for Neural Relation Extractioncited by 62
  19. 2019Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networkscited by 61
  20. 2020Contrastive Triple Extraction with Generative Transformercited by 44
  21. 2020Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphscited by 39
  22. 2021Curriculum-Meta Learning for Order-Robust Continual Relation Extractioncited by 34
  23. 2020GDPNet: Refining Latent Multi-View Graph for Relation Extractioncited by 29
  24. 2020RECON: Relation Extraction using Knowledge Graph Context in a Graph Neural Networkcited by 28
  25. 2021A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extractioncited by 26
  26. 2019NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extractioncited by 24
  27. 2019Neural Snowball for Few-Shot Relation Learningcited by 22
  28. 2020Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extractioncited by 18
  29. 2020Logic-guided Semantic Representation Learning for Zero-Shot Relation Classificationcited by 18
  30. 2020Finding Influential Instances for Distantly Supervised Relation Extractioncited by 13
  31. 2018Attention-Based Capsule Networks with Dynamic Routing for Relation Extractioncited by 10
  32. 2019Improving Relation Extraction with Knowledge-attentioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 45; 2023: 65; 2024: 42; 2025: 15; 2026: 4.

180 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  3. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  4. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  5. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  6. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  7. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  8. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  9. 2023Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!cited by 119
  10. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  11. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  12. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
38 more
  1. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 97
  2. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  3. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  4. 2023Large Language Models and Knowledge Graphs: Opportunities and Challengescited by 71
  5. 2022OntoProtein: Protein Pretraining With Gene Ontology Embeddingcited by 70
  6. 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
  7. 2024Injecting New Knowledge into Large Language Models via Supervised Fine-Tuningcited by 56
  8. 2022Generative Knowledge Graph Construction: A Reviewcited by 49
  9. 2026SkillNet: Create, Evaluate, and Connect AI Skillscited by 48
  10. 2022Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completioncited by 44
  11. 2023Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractorscited by 44
  12. 2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
  13. 2022Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learningcited by 36
  14. 2022HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classificationcited by 31
  15. 2022Summarization as Indirect Supervision for Relation Extractioncited by 31
  16. 2023Large Language Model Instruction Following: A Survey of Progresses and Challengescited by 31
  17. 2024Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?cited by 31
  18. 2022Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysiscited by 30
  19. 2023How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?cited by 30
  20. 2023Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Modelscited by 30
  21. 2023Biomedical knowledge graph-optimized prompt generation for large language modelscited by 30
  22. 2023InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understandingcited by 29
  23. 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
  24. 2022Neuro-Symbolic Procedural Planning with Commonsense Promptingcited by 28
  25. 2022Ontology-enhanced Prompt-tuning for Few-shot Learningcited by 24
  26. 2022LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learningcited by 24
  27. 2022Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detectioncited by 23
  28. 2023Editing Language Model-based Knowledge Graph Embeddingscited by 23
  29. 2023Multitask Instruction-based Prompting for Fallacy Recognitioncited by 22
  30. 2023A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontierscited by 21
  31. 2024On the Way to LLM Personalization: Learning to Remember User Conversationscited by 21
  32. 2022UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQLcited by 20
  33. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  34. 2023When Do Program-of-Thoughts Work for Reasoning?cited by 19
  35. 2022Towards Unified Prompt Tuning for Few-shot Text Classificationcited by 18
  36. 2022Tele-Knowledge Pre-training for Fault Analysiscited by 18
  37. 2023SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understandingcited by 18
  38. 2022DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Populationcited by 17

50 most cited of 180 All 180 (JSON)