Towards Realistic Few-Shot Relation Extraction: A New Meta Dataset and Evaluation
Not on the map. Why?
Abstract (arXiv)
We introduce a meta dataset for few-shot relation extraction, which includes two datasets derived from existing supervised relation extraction datasets NYT29 (Takanobu et al., 2019; Nayak and Ng, 2020) and WIKIDATA (Sorokin and Gurevych, 2017) as well as a few-shot form of the TACRED dataset (Sabo et al., 2021). Importantly, all these few-shot datasets were generated under realistic assumptions such as: the test relations are different from any relations a model might have seen before, limited training data, and a preponderance of candidate relation mentions that do not correspond to any of the relations of interest. Using this large resource, we conduct a comprehensive evaluation of six recent few-shot relation extraction methods, and observe that no method comes out as a clear winner. Further, the overall performance on this task is low, indicating substantial need for future research. We release all versions of the data, i.e., both supervised and few-shot, for future research.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 72 references that are in the map. First posted on arXiv 5 Apr 2024.
Builds on
30 of 72 references.
References · 30
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2018Deep contextualized word representationscited by 2,285
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2016Enriching Word Vectors with Subword Informationcited by 1,578
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
- 2016Neural Architectures for Named Entity Recognitioncited by 548
- 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
- 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
18 more
- 2016End-to-End Relation Extraction using LSTMs on Sequences and Tree Structurescited by 161
- 2018Graph Convolution over Pruned Dependency Trees Improves Relation Extractioncited by 146
- 2017Learned in Translation: Contextualized Word Vectorscited by 104
- 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
- 2021An Improved Baseline for Sentence-level Relation Extractioncited by 80
- 2019Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extractioncited by 73
- 2021Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extractioncited by 73
- 2018A Hierarchical Framework for Relation Extraction with Reinforcement Learningcited by 49
- 2015Relation Classification via Recurrent Neural Networkcited by 46
- 2023Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractorscited by 44
- 2022Summarization as Indirect Supervision for Relation Extractioncited by 31
- 2021Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemescited by 27
- 2019NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extractioncited by 24
- 2022What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classificationcited by 17
- 2020Relation Classification as Two-way Span-Predictioncited by 9
- 2023Knowledge Base Completion for Long-Tail Entitiescited by 7
- 2024Best of Both Worlds: A Pliable and Generalizable Neuro-Symbolic Approach for Relation Classificationcited by 6
- 2022From Examples to Rules: Neural Guided Rule Synthesis for Information Extractioncited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.