# Generative Prompt Tuning for Relation Classification

arXiv: https://arxiv.org/abs/2210.12435 · HTML: https://mlsanity.com/paper/2210.12435/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 58 references that are in the map.
- First posted on arXiv 22 Oct 2022.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [WARP: Word-level Adversarial ReProgramming](https://mlsanity.com/paper/2101.00121/) (2021)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2108.13161/) (2021)
- [Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification](https://mlsanity.com/paper/2010.13641/) (2020)
- [Improving and Simplifying Pattern Exploiting Training](https://mlsanity.com/paper/2103.11955/) (2021)
- [TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task](https://mlsanity.com/paper/2004.14855/) (2020)
- [An Improved Baseline for Sentence-level Relation Extraction](https://mlsanity.com/paper/2102.01373/) (2021)
- [Re-TACRED: Addressing Shortcomings of the TACRED Dataset](https://mlsanity.com/paper/2104.08398/) (2021)
- [Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction](https://mlsanity.com/paper/1911.09886/) (2019)

## Built on by (most cited first, 13 of 13)

- [How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?](https://mlsanity.com/paper/2305.01555/) (2023)
- [Retrieval-Augmented Generation-based Relation Extraction](https://mlsanity.com/paper/2404.13397/) (2024)
- [Multilingual Relation Classification via Efficient and Effective Prompting](https://mlsanity.com/paper/2210.13838/) (2022)
- [STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models](https://mlsanity.com/paper/2305.15090/) (2023)
- [Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-based Causal Discovery](https://mlsanity.com/paper/2407.18752/) (2024)
- [Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training](https://mlsanity.com/paper/2505.12236/) (2025)
- [A Few-Shot Approach for Relation Extraction Domain Adaptation using Large Language Models](https://mlsanity.com/paper/2408.02377/) (2024)
- [Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction](https://mlsanity.com/paper/2308.02103/) (2023)
- [Advancing Relation Extraction through Language Probing with Exemplars from Set Co-Expansion](https://mlsanity.com/paper/2308.11720/) (2023)
- [Empowering Few-Shot Relation Extraction with The Integration of Traditional RE Methods and Large Language Models](https://mlsanity.com/paper/2407.08967/) (2024)
- [Maximizing Relation Extraction Potential: A Data-Centric Study to Unveil Challenges and Opportunities](https://mlsanity.com/paper/2409.04934/) (2024)
- [CPTuning: Contrastive Prompt Tuning for Generative Relation Extraction](https://mlsanity.com/paper/2501.02196/) (2025)
- [PiPMRE: A Pipeline Based on Language Model for Medical Relation Extraction](https://mlsanity.com/paper/2609.02896/) (2026)

## Abstract (arXiv)

Using prompts to explore the knowledge contained within pre-trained language models for downstream tasks has now become an active topic. Current prompt tuning methods mostly convert the downstream tasks to masked language modeling problems by adding cloze-style phrases and mapping all labels to verbalizations with fixed length, which has proven effective for tasks with simple label spaces. However, when applied to relation classification exhibiting complex label spaces, vanilla prompt tuning methods may struggle with label verbalizations with arbitrary lengths due to rigid prompt restrictions. Inspired by the text infilling task for pre-training generative models that can flexibly predict missing spans, we propose a novel generative prompt tuning method to reformulate relation classification as an infilling problem, which frees our approach from limitations of current prompt based approaches and thus fully exploits rich semantics of entity and relation types. In addition, we design entity-guided decoding and discriminative relation scoring to generate and align relations effectively and efficiently during inference. Extensive experiments under fully supervised settings and low-resource settings demonstrate the effectiveness of our approach.