mlsanityData to 14 Oct 2026

Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling

arXiv 2104.07704 ↗First posted 15 Apr 2021cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent models have shown that incorporating syntactic knowledge into the semantic role labelling (SRL) task leads to a significant improvement. In this paper, we propose Syntax-aware Graph-to-Graph Transformer (SynG2G-Tr) model, which encodes the syntactic structure using a novel way to input graph relations as embeddings, directly into the self-attention mechanism of Transformer. This approach adds a soft bias towards attention patterns that follow the syntactic structure but also allows the model to use this information to learn alternative patterns. We evaluate our model on both span-based and dependency-based SRL datasets, and outperform previous alternative methods in both in-domain and out-of-domain settings, on CoNLL 2005 and CoNLL 2009 datasets.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 33 of its 76 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

33 of 76 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2016Layer Normalizationcited by 5,669
  5. 2018Deep contextualized word representationscited by 2,290
  6. 2018Self-Attention with Relative Position Representationscited by 1,120
  7. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  8. 2017Encoding Sentences with Graph Convolutional Networks for Semantic Role Labelingcited by 137
  9. 2019Semantics-aware BERT for Language Understandingcited by 112
  10. 2020Improve Transformer Models with Better Relative Position Embeddingscited by 73
  11. 2018Linguistically-Informed Self-Attention for Semantic Role Labelingcited by 71
  12. 2019Head-Driven Phrase Structure Grammar Parsing on Penn Treebankcited by 43
21 more
  1. 2018Jointly Predicting Predicates and Arguments in Neural Semantic Role Labelingcited by 34
  2. 2017Deep Semantic Role Labeling with Self-Attentioncited by 23
  3. 2019Dependency or Span, End-to-End Uniform Semantic Role Labelingcited by 22
  4. 2020Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinementcited by 20
  5. 2017Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classificationcited by 17
  6. 2016Neural Semantic Role Labeling with Dependency Path Embeddingscited by 15
  7. 2018A Span Selection Model for Semantic Role Labelingcited by 15
  8. 2018A Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?cited by 13
  9. 2019Parsing All: Syntax and Semantics, Dependencies and Spanscited by 13
  10. 2019Graph-to-Graph Transformer for Transition-based Dependency Parsingcited by 13
  11. 2017A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labelingcited by 10
  12. 2019Syntax-aware Neural Semantic Role Labelingcited by 8
  13. 2019Syntax-aware Multilingual Semantic Role Labelingcited by 7
  14. 2020Syntax Role for Neural Semantic Role Labelingcited by 7
  15. 2019Syntax-aware Neural Semantic Role Labeling with Supertagscited by 6
  16. 2019Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Taskcited by 6
  17. 2020Improved Semantic Role Labeling using Parameterized Neighborhood Memory Adaptationcited by 6
  18. 2019Semantic Role Labeling with Iterative Structure Refinementcited by 5
  19. 2020Adaptive Convolution for Semantic Role Labelingcited by 4
  20. 2017Fast and Accurate Neural Word Segmentation for Chinesecited by 3
  21. 2019Capturing Argument Interaction in Semantic Role Labeling with Capsule Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 4.

8 papers.

  1. 2022RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Questioncited by 17
  2. 2021Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures Inside Argumentscited by 14
  3. 2023Ensemble Distillation for Unsupervised Constituency Parsingcited by 6
  4. 2023Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoderscited by 6
  5. 2023Transformers as Graph-to-Graph Modelscited by 5
  6. 2021An MRC Framework for Semantic Role Labelingcited by 3
  7. 2022Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformerscited by 2
  8. 2023GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extractioncited by 1