Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters
Not on the map. Why?
Abstract (arXiv)
Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to fit the PLM tokenization requirements. Nevertheless, this transformation jeopardizes the structural integrity of the graph and is therefore detrimental to its resulting representation. To overcome this issue, Ribeiro et al. have recently proposed StructAdapt, a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text, and, in parallel, we examine the robustness of StructAdapt. Through ablation studies, graph attack and link prediction, we reveal that RPE might be partially encoding input graphs. We suggest further research regarding the role of RPE will provide valuable insights for Graph-to-Text generation.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 12 Feb 2023.
Builds on
23 of 41 references.
References · 23
- 2017Attention Is All You Needcited by 59,932
- 2017Graph Attention Networkscited by 8,854
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
- 2018Self-Attention with Relative Position Representationscited by 1,112
- 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
- 2021Position Information in Transformers: An Overviewcited by 103
- 2018Graph-to-Sequence Learning using Gated Graph Neural Networkscited by 61
- 2017Neural AMR: Sequence-to-Sequence Models for Parsing and Generationcited by 56
11 more
- 2018A Graph-to-Sequence Model for AMR-to-Text Generationcited by 56
- 2020What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encodingcited by 52
- 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
- 2020AMR Parsing via Graph-Sequence Iterative Inferencecited by 47
- 2019Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learningcited by 29
- 2020Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMRcited by 25
- 2019Enhancing AMR-to-Text Generation with Dual Graph Representationscited by 24
- 2019Structural Neural Encoders for AMR-to-text Generationcited by 19
- 2020Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generationcited by 8
- 2016AMR-to-text generation as a Traveling Salesman Problemcited by 4
- 2018Factorising AMR generation through syntaxcited by 4
Built on by
Citing papers per year: 2025: 2.
2 papers.