# Enriched Attention for Robust Relation Extraction

arXiv: https://arxiv.org/abs/2104.10899 · HTML: https://mlsanity.com/paper/2104.10899/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [Entity-Relation Extraction as Multi-Turn Question Answering](https://mlsanity.com/paper/1905.05529/) (2019)
- [A General Framework for Information Extraction using Dynamic Span Graphs](https://mlsanity.com/paper/1904.03296/) (2019)
- [Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations](https://mlsanity.com/paper/1909.10681/) (2019)
- [Attention Guided Graph Convolutional Networks for Relation Extraction](https://mlsanity.com/paper/1906.07510/) (2019)
- [BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages](https://mlsanity.com/paper/1710.02187/) (2017)
- [Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path](https://mlsanity.com/paper/1508.03720/) (2015)
- [CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases](https://mlsanity.com/paper/1610.08763/) (2016)
- [Improving Relation Extraction by Pre-trained Language Representations](https://mlsanity.com/paper/1906.03088/) (2019)
- [Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data](https://mlsanity.com/paper/1906.08931/) (2019)
- [Learning Distributed Representations of Texts and Entities from Knowledge Base](https://mlsanity.com/paper/1705.02494/) (2017)
- [A Walk-based Model on Entity Graphs for Relation Extraction](https://mlsanity.com/paper/1902.07023/) (2019)
- [Exploring Different Dimensions of Attention for Uncertainty Detection](https://mlsanity.com/paper/1612.06549/) (2016)
- [Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structures](https://mlsanity.com/paper/1707.01075/) (2017)
- [Corpus-level Fine-grained Entity Typing](https://mlsanity.com/paper/1708.02275/) (2017)
- [Improving Relation Extraction with Knowledge-attention](https://mlsanity.com/paper/1910.02724/) (2019)
- [Type-aware Convolutional Neural Networks for Slot Filling](https://mlsanity.com/paper/1910.00546/) (2019)
- [Combining Neural Networks and Log-linear Models to Improve Relation Extraction](https://mlsanity.com/paper/1511.05926/) (2015)

## Built on by (most cited first, 1 of 1)

- [Noise in Relation Classification Dataset TACRED: Characterization and Reduction](https://mlsanity.com/paper/2311.12298/) (2023)

## Abstract (arXiv)

The performance of relation extraction models has increased considerably with the rise of neural networks. However, a key issue of neural relation extraction is robustness: the models do not scale well to long sentences with multiple entities and relations. In this work, we address this problem with an enriched attention mechanism. Attention allows the model to focus on parts of the input sentence that are relevant to relation extraction. We propose to enrich the attention function with features modeling knowledge about the relation arguments and the shortest dependency path between them. Thus, for different relation arguments, the model can pay attention to different parts of the sentence. Our model outperforms prior work using comparable setups on two popular benchmarks, and our analysis confirms that it indeed scales to long sentences with many entities.