# Deep Semantic Role Labeling with Self-Attention

arXiv: https://arxiv.org/abs/1712.01586 · HTML: https://mlsanity.com/paper/1712.01586/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2017.

## Built on by (most cited first, 23 of 23)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Pushing the Limits of ChatGPT on NLP Tasks](https://mlsanity.com/paper/2306.09719/) (2023)
- [GSRFormer: Grounded Situation Recognition Transformer with Alternate Semantic Attention Refinement](https://mlsanity.com/paper/2208.08965/) (2022)
- [Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Arguments](https://mlsanity.com/paper/2110.06865/) (2021)
- [Does Head Label Help for Long-Tailed Multi-Label Text Classification](https://mlsanity.com/paper/2101.09704/) (2021)
- [Linguistically-Enriched and Context-Aware Zero-shot Slot Filling](https://mlsanity.com/paper/2101.06514/) (2021)
- [Conversational Semantic Role Labeling](https://mlsanity.com/paper/2104.04947/) (2021)
- [Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling](https://mlsanity.com/paper/2104.07704/) (2021)
- [Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data](https://mlsanity.com/paper/2305.03827/) (2023)
- [Are there identifiable structural parts in the sentence embedding whole?](https://mlsanity.com/paper/2406.16563/) (2024)
- [Improving Aspect-Based Sentiment with End-to-End Semantic Role Labeling Model](https://mlsanity.com/paper/2307.14785/) (2023)
- [Knowledge Graph Anchored Information-Extraction for Domain-Specific Insights](https://mlsanity.com/paper/2104.08936/) (2021)
- [An MRC Framework for Semantic Role Labeling](https://mlsanity.com/paper/2109.06660/) (2021)
- [Topic Driven Adaptive Network for Cross-Domain Sentiment Classification](https://mlsanity.com/paper/2111.14094/) (2021)
- [Toward Open-domain Slot Filling via Self-supervised Co-training](https://mlsanity.com/paper/2303.13801/) (2023)
- [Distantly-Supervised Joint Extraction with Noise-Robust Learning](https://mlsanity.com/paper/2310.04994/) (2023)
- [Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks](https://mlsanity.com/paper/2102.04610/) (2021)
- [Extracting Variable-Depth Logical Document Hierarchy from Long Documents: Method, Evaluation, and Application](https://mlsanity.com/paper/2105.09297/) (2021)
- [Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing](https://mlsanity.com/paper/2112.02970/) (2021)
- [Introducing "Forecast Utterance" for Conversational Data Science](https://mlsanity.com/paper/2309.03877/) (2023)
- [AED: An black-box NLP classifier model attacker](https://mlsanity.com/paper/2112.11660/) (2021)
- [Attention Mechanism with Energy-Friendly Operations](https://mlsanity.com/paper/2204.13353/) (2022)
- [SemSketches-2021: experimenting with the machine processing of the pilot semantic sketches corpus](https://mlsanity.com/paper/2505.17704/) (2025)

## Abstract (arXiv)

Semantic Role Labeling (SRL) is believed to be a crucial step towards natural language understanding and has been widely studied. Recent years, end-to-end SRL with recurrent neural networks (RNN) has gained increasing attention. However, it remains a major challenge for RNNs to handle structural information and long range dependencies. In this paper, we present a simple and effective architecture for SRL which aims to address these problems. Our model is based on self-attention which can directly capture the relationships between two tokens regardless of their distance. Our single model achieves F1=83.4 on the CoNLL-2005 shared task dataset and F1=82.7 on the CoNLL-2012 shared task dataset, which outperforms the previous state-of-the-art results by 1.8 and 1.0 F1 score respectively. Besides, our model is computationally efficient, and the parsing speed is 50K tokens per second on a single Titan X GPU.