# Modeling Graph Structure in Transformer for Better AMR-to-Text Generation

arXiv: https://arxiv.org/abs/1909.00136 · HTML: https://mlsanity.com/paper/1909.00136/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2019.

## Built on by (most cited first, 30 of 35)

- [Transformer for Graphs: An Overview from Architecture Perspective](https://mlsanity.com/paper/2202.08455/) (2022)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs](https://mlsanity.com/paper/2106.10502/) (2021)
- [Graph Pre-training for AMR Parsing and Generation](https://mlsanity.com/paper/2203.07836/) (2022)
- [AMR Parsing with Action-Pointer Transformer](https://mlsanity.com/paper/2104.14674/) (2021)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [Semantic Representation for Dialogue Modeling](https://mlsanity.com/paper/2105.10188/) (2021)
- [GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation](https://mlsanity.com/paper/2204.06674/) (2022)
- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [Unifying Structured Data as Graph for Data-to-Text Pre-Training](https://mlsanity.com/paper/2401.01183/) (2024)
- [Graphine: A Dataset for Graph-aware Terminology Definition Generation](https://mlsanity.com/paper/2109.04018/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Improving Medical Dialogue Generation with Abstract Meaning Representations](https://mlsanity.com/paper/2309.10608/) (2023)
- [Llamipa: An Incremental Discourse Parser](https://mlsanity.com/paper/2406.18256/) (2024)
- [Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions](https://mlsanity.com/paper/2203.12235/) (2022)
- [Semantic-based Pre-training for Dialogue Understanding](https://mlsanity.com/paper/2209.09146/) (2022)
- [CATS: A Pragmatic Chinese Answer-to-Sequence Dataset with Large Scale and High Quality](https://mlsanity.com/paper/2306.11477/) (2023)
- [Can LLMs Interpret and Leverage Structured Linguistic Representations? A Case Study with AMRs](https://mlsanity.com/paper/2504.04745/) (2025)
- [Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph](https://mlsanity.com/paper/2112.00503/) (2021)
- [Graph-to-Text Generation with Dynamic Structure Pruning](https://mlsanity.com/paper/2209.07258/) (2022)
- [SGRAM: Improving Scene Graph Parsing via Abstract Meaning Representation](https://mlsanity.com/paper/2210.08675/) (2022)
- [Cross-domain Generalization for AMR Parsing](https://mlsanity.com/paper/2210.12445/) (2022)
- [Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation](https://mlsanity.com/paper/2404.01129/) (2024)
- [Survey of Abstract Meaning Representation: Then, Now, Future](https://mlsanity.com/paper/2505.03229/) (2025)
- [A Graph Transformer-Driven Approach for Network Robustness Learning](https://mlsanity.com/paper/2306.06913/) (2023)
- [Reranking for Natural Language Generation from Logical Forms: A Study based on Large Language Models](https://mlsanity.com/paper/2309.12294/) (2023)
- [To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graph](https://mlsanity.com/paper/2101.06397/) (2021)

## Abstract (arXiv)

Recent studies on AMR-to-text generation often formalize the task as a sequence-to-sequence (seq2seq) learning problem by converting an Abstract Meaning Representation (AMR) graph into a word sequence. Graph structures are further modeled into the seq2seq framework in order to utilize the structural information in the AMR graphs. However, previous approaches only consider the relations between directly connected concepts while ignoring the rich structure in AMR graphs. In this paper we eliminate such a strong limitation and propose a novel structure-aware self-attention approach to better modeling the relations between indirectly connected concepts in the state-of-the-art seq2seq model, i.e., the Transformer. In particular, a few different methods are explored to learn structural representations between two concepts. Experimental results on English AMR benchmark datasets show that our approach significantly outperforms the state of the art with 29.66 and 31.82 BLEU scores on LDC2015E86 and LDC2017T10, respectively. To the best of our knowledge, these are the best results achieved so far by supervised models on the benchmarks.