# Pre-training to Match for Unified Low-shot Relation Extraction

arXiv: https://arxiv.org/abs/2203.12274 · HTML: https://mlsanity.com/paper/2203.12274/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Few-Shot Learning with Graph Neural Networks](https://mlsanity.com/paper/1711.04043/) (2017)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [Zero-Shot Learning Through Cross-Modal Transfer](https://mlsanity.com/paper/1301.3666/) (2013)
- [FewRel 2.0: Towards More Challenging Few-Shot Relation Classification](https://mlsanity.com/paper/1910.07124/) (2019)
- [Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction](https://mlsanity.com/paper/2109.03659/) (2021)
- [FLEX: Unifying Evaluation for Few-Shot NLP](https://mlsanity.com/paper/2107.07170/) (2021)
- [Neural Open Information Extraction](https://mlsanity.com/paper/1805.04270/) (2018)
- [ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning](https://mlsanity.com/paper/2104.04697/) (2021)
- [Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification](https://mlsanity.com/paper/1906.06678/) (2019)
- [MapRE: An Effective Semantic Mapping Approach for Low-resource Relation Extraction](https://mlsanity.com/paper/2109.04108/) (2021)
- [Element Intervention for Open Relation Extraction](https://mlsanity.com/paper/2106.09558/) (2021)

## Built on by (most cited first, 8 of 8)

- [GPT-RE: In-context Learning for Relation Extraction using Large Language Models](https://mlsanity.com/paper/2305.02105/) (2023)
- [Universal Information Extraction as Unified Semantic Matching](https://mlsanity.com/paper/2301.03282/) (2023)
- [Better Few-Shot Relation Extraction with Label Prompt Dropout](https://mlsanity.com/paper/2210.13733/) (2022)
- [ProtoEM: A Prototype-Enhanced Matching Framework for Event Relation Extraction](https://mlsanity.com/paper/2309.12892/) (2023)
- [Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction](https://mlsanity.com/paper/2312.12021/) (2023)
- [Learning from Semi-Factuals: A Debiased and Semantic-Aware Framework for Generalized Relation Discovery](https://mlsanity.com/paper/2401.06327/) (2024)
- [Many-Class Text Classification with Matching](https://mlsanity.com/paper/2205.11409/) (2022)
- [Few-shot Open Relation Extraction with Gaussian Prototype and Adaptive Margin](https://mlsanity.com/paper/2410.20320/) (2024)

## Abstract (arXiv)

Low-shot relation extraction~(RE) aims to recognize novel relations with very few or even no samples, which is critical in real scenario application. Few-shot and zero-shot RE are two representative low-shot RE tasks, which seem to be with similar target but require totally different underlying abilities. In this paper, we propose Multi-Choice Matching Networks to unify low-shot relation extraction. To fill in the gap between zero-shot and few-shot RE, we propose the triplet-paraphrase meta-training, which leverages triplet paraphrase to pre-train zero-shot label matching ability and uses meta-learning paradigm to learn few-shot instance summarizing ability. Experimental results on three different low-shot RE tasks show that the proposed method outperforms strong baselines by a large margin, and achieve the best performance on few-shot RE leaderboard.