# Position-aware Self-attention with Relative Positional Encodings for Slot Filling

arXiv: https://arxiv.org/abs/1807.03052 · HTML: https://mlsanity.com/paper/1807.03052/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2018.

## Built on by (most cited first, 5 of 5)

- [Prototypical Representation Learning for Relation Extraction](https://mlsanity.com/paper/2103.11647/) (2021)
- [Modeling Spatial Trajectories using Coarse-Grained Smartphone Logs](https://mlsanity.com/paper/2208.13775/) (2022)
- [Modeling Time-Series and Spatial Data for Recommendations and Other Applications](https://mlsanity.com/paper/2212.13259/) (2022)
- [Embeddings for Tabular Data: A Survey](https://mlsanity.com/paper/2302.11777/) (2023)
- [Entity-Aware Self-Attention and Contextualized GCN for Enhanced Relation Extraction in Long Sentences](https://mlsanity.com/paper/2409.13755/) (2024)

## Abstract (arXiv)

This paper describes how to apply self-attention with relative positional encodings to the task of relation extraction. We propose to use the self-attention encoder layer together with an additional position-aware attention layer that takes into account positions of the query and the object in the sentence. The self-attention encoder also uses a custom implementation of relative positional encodings which allow each word in the sentence to take into account its left and right context. The evaluation of the model is done on the TACRED dataset. The proposed model relies only on attention (no recurrent or convolutional layers are used), while improving performance w.r.t. the previous state of the art.