# Position-Aware Self-Attention based Neural Sequence Labeling

arXiv: https://arxiv.org/abs/1908.09128 · HTML: https://mlsanity.com/paper/1908.09128/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2019.

## Built on by (most cited first, 6 of 6)

- [Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models](https://mlsanity.com/paper/2106.05505/) (2021)
- [Position-based Prompting for Health Outcome Generation](https://mlsanity.com/paper/2204.03489/) (2022)
- [Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision](https://mlsanity.com/paper/2404.06723/) (2024)
- [Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records](https://mlsanity.com/paper/2403.04012/) (2024)
- [Improving Pseudo Labels with Global-Local Denoising Framework for Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2406.01213/) (2024)
- [Exploiting Global Contextual Information for Document-level Named Entity Recognition](https://mlsanity.com/paper/2106.00887/) (2021)

## Abstract (arXiv)

Sequence labeling is a fundamental task in natural language processing and has been widely studied. Recently, RNN-based sequence labeling models have increasingly gained attentions. Despite superior performance achieved by learning the long short-term (i.e., successive) dependencies, the way of sequentially processing inputs might limit the ability to capture the non-continuous relations over tokens within a sentence. To tackle the problem, we focus on how to effectively model successive and discrete dependencies of each token for enhancing the sequence labeling performance. Specifically, we propose an innovative attention-based model (called position-aware selfattention, i.e., PSA) as well as a well-designed self-attentional context fusion layer within a neural network architecture, to explore the positional information of an input sequence for capturing the latent relations among tokens. Extensive experiments on three classical tasks in sequence labeling domain, i.e., partof-speech (POS) tagging, named entity recognition (NER) and phrase chunking, demonstrate our proposed model outperforms the state-of-the-arts without any external knowledge, in terms of various metrics.