# Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters

arXiv: https://arxiv.org/abs/2302.05900 · HTML: https://mlsanity.com/paper/2302.05900/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 12 Feb 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://mlsanity.com/paper/2007.08426/) (2020)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [Graph-to-Sequence Learning using Gated Graph Neural Networks](https://mlsanity.com/paper/1806.09835/) (2018)
- [Neural AMR: Sequence-to-Sequence Models for Parsing and Generation](https://mlsanity.com/paper/1704.08381/) (2017)
- [A Graph-to-Sequence Model for AMR-to-Text Generation](https://mlsanity.com/paper/1805.02473/) (2018)
- [What Do Position Embeddings Learn? An Empirical Study of Pre-Trained Language Model Positional Encoding](https://mlsanity.com/paper/2010.04903/) (2020)
- [Structural Adapters in Pretrained Language Models for AMR-to-text Generation](https://mlsanity.com/paper/2103.09120/) (2021)
- [AMR Parsing via Graph-Sequence Iterative Inference](https://mlsanity.com/paper/2004.05572/) (2020)
- [Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning](https://mlsanity.com/paper/1908.05957/) (2019)
- [Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR](https://mlsanity.com/paper/2008.08896/) (2020)
- [Enhancing AMR-to-Text Generation with Dual Graph Representations](https://mlsanity.com/paper/1909.00352/) (2019)
- [Structural Neural Encoders for AMR-to-text Generation](https://mlsanity.com/paper/1903.11410/) (2019)
- [Lightweight, Dynamic Graph Convolutional Networks for AMR-to-Text Generation](https://mlsanity.com/paper/2010.04383/) (2020)
- [AMR-to-text generation as a Traveling Salesman Problem](https://mlsanity.com/paper/1609.07451/) (2016)
- [Factorising AMR generation through syntax](https://mlsanity.com/paper/1804.07707/) (2018)

## Built on by (most cited first, 2 of 2)

- [Survey of Abstract Meaning Representation: Then, Now, Future](https://mlsanity.com/paper/2505.03229/) (2025)
- [HGAdapter: Hypergraph-based Adapters in Language Models for Code Summarization and Clone Detection](https://mlsanity.com/paper/2510.17591/) (2025)

## Abstract (arXiv)

Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to fit the PLM tokenization requirements. Nevertheless, this transformation jeopardizes the structural integrity of the graph and is therefore detrimental to its resulting representation. To overcome this issue, Ribeiro et al. have recently proposed StructAdapt, a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text, and, in parallel, we examine the robustness of StructAdapt. Through ablation studies, graph attack and link prediction, we reveal that RPE might be partially encoding input graphs. We suggest further research regarding the role of RPE will provide valuable insights for Graph-to-Text generation.