mlsanityData to 7 Oct 2026

PTR: Prompt Tuning with Rules for Text Classification

arXiv 2105.11259 ↗First posted 24 May 2021cs.CLcited by 238

Not on the map. Why?

Abstract (arXiv)

Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.

Key facts. Cited by 238 papers in the map (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

40 of 54 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  11. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  12. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
28 more
  1. 2021GPT Understands, Toocited by 715
  2. 2019Text Summarization with Pretrained Encoderscited by 553
  3. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  4. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  5. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  6. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  7. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  8. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  9. 2019Knowledge Enhanced Contextual Word Representationscited by 294
  10. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  11. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  12. 2018A Simple Method for Commonsense Reasoningcited by 223
  13. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 192
  14. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  15. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  16. 2021Few-NERD: A Few-Shot Named Entity Recognition Datasetcited by 150
  17. 2018Graph Convolution over Pruned Dependency Trees Improves Relation Extractioncited by 146
  18. 2019Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Modelscited by 141
  19. 2020Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classificationcited by 98
  20. 2020TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Taskcited by 96
  21. 2021An Improved Baseline for Sentence-level Relation Extractioncited by 80
  22. 2020Learning from Context or Names? An Empirical Study on Neural Relation Extractioncited by 75
  23. 2021Re-TACRED: Addressing Shortcomings of the TACRED Datasetcited by 73
  24. 2019OpenNRE: An Open and Extensible Toolkit for Neural Relation Extractioncited by 62
  25. 2020Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inferencecited by 47
  26. 2021Prototypical Representation Learning for Relation Extractioncited by 33
  27. 2016CoType: Joint Extraction of Typed Entities and Relations with Knowledge Basescited by 28
  28. 2016Neural Architectures for Fine-grained Entity Type Classificationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 76; 2023: 76; 2024: 42; 2025: 12; 2026: 6.

238 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  4. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  5. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  6. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  7. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  8. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  9. 2022Prompt Distribution Learningcited by 235
  10. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  11. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  12. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
38 more
  1. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  2. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  3. 2022A Review on Language Models as Knowledge Basescited by 156
  4. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  5. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  6. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  7. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  8. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  9. 2022MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuningcited by 109
  10. 2023Text Classification via Large Language Modelscited by 104
  11. 2021LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5cited by 103
  12. 2024LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuningcited by 88
  13. 2022FedPrompt: Communication-Efficient and Privacy Preserving Prompt Tuning in Federated Learningcited by 86
  14. 2022PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discoverycited by 85
  15. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  16. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  17. 2021Template-free Prompt Tuning for Few-shot NERcited by 73
  18. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  19. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  20. 2021Prompt-Learning for Fine-Grained Entity Typingcited by 62
  21. 2022ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalizationcited by 54
  22. 2022Prototypical Verbalizer for Prompt-based Few-shot Tuningcited by 53
  23. 2021CPM-2: Large-scale Cost-effective Pre-trained Language Modelscited by 52
  24. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  25. 2021Packed Levitated Marker for Entity and Relation Extractioncited by 49
  26. 2023Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Modelscited by 49
  27. 2023TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Taskscited by 48
  28. 2023Learning from models beyond fine-tuningcited by 47
  29. 2022Continual Prompt Tuning for Dialog State Trackingcited by 44
  30. 2023Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractorscited by 44
  31. 2023Prompting Large Language Models With the Socratic Methodcited by 43
  32. 2022No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligencecited by 41
  33. 2021Paradigm Shift in Natural Language Processingcited by 39
  34. 2023Black Box Few-Shot Adaptation for Vision-Language modelscited by 39
  35. 2023ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relationscited by 39
  36. 2022GPS: Genetic Prompt Search for Efficient Few-shot Learningcited by 37
  37. 2022Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learningcited by 36
  38. 2022PromptBoosting: Black-Box Text Classification with Ten Forward Passescited by 36

50 most cited of 238 All 238 (JSON)