# Two Training Strategies for Improving Relation Extraction over Universal Graph

arXiv: https://arxiv.org/abs/2102.06540 · HTML: https://mlsanity.com/paper/2102.06540/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [Distant Supervision for Relation Extraction beyond the Sentence Boundary](https://mlsanity.com/paper/1609.04873/) (2016)
- [RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information](https://mlsanity.com/paper/1812.04361/) (2018)
- [Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks](https://mlsanity.com/paper/1607.01426/) (2016)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention](https://mlsanity.com/paper/1804.06987/) (2018)
- [Compositional Vector Space Models for Knowledge Base Completion](https://mlsanity.com/paper/1504.06662/) (2015)
- [Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions](https://mlsanity.com/paper/1904.00143/) (2019)
- [Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks](https://mlsanity.com/paper/1704.08384/) (2017)
- [Incorporating Relation Paths in Neural Relation Extraction](https://mlsanity.com/paper/1609.07479/) (2016)
- [Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1809.00699/) (2018)
- [Leveraging 2-hop Distant Supervision from Table Entity Pairs for Relation Extraction](https://mlsanity.com/paper/1909.06007/) (2019)

## Built on by (most cited first, 1 of 1)

- [Cross-stitching Text and Knowledge Graph Encoders for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2211.01432/) (2022)

## Abstract (arXiv)

This paper explores how the Distantly Supervised Relation Extraction (DS-RE) can benefit from the use of a Universal Graph (UG), the combination of a Knowledge Graph (KG) and a large-scale text collection. A straightforward extension of a current state-of-the-art neural model for DS-RE with a UG may lead to degradation in performance. We first report that this degradation is associated with the difficulty in learning a UG and then propose two training strategies: (1) Path Type Adaptive Pretraining, which sequentially trains the model with different types of UG paths so as to prevent the reliance on a single type of UG path; and (2) Complexity Ranking Guided Attention mechanism, which restricts the attention span according to the complexity of a UG path so as to force the model to extract features not only from simple UG paths but also from complex ones. Experimental results on both biomedical and NYT10 datasets prove the robustness of our methods and achieve a new state-of-the-art result on the NYT10 dataset. The code and datasets used in this paper are available at https://github.com/baodaiqin/UGDSRE.