# Graph Transformer for Graph-to-Sequence Learning

arXiv: https://arxiv.org/abs/1911.07470 · HTML: https://mlsanity.com/paper/1911.07470/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 30 of 82)

- [Do Transformers Really Perform Bad for Graph Representation?](https://mlsanity.com/paper/2106.05234/) (2021)
- [Global Self-Attention as a Replacement for Graph Convolution](https://mlsanity.com/paper/2108.03348/) (2021)
- [Transformer for Graphs: An Overview from Architecture Perspective](https://mlsanity.com/paper/2202.08455/) (2022)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction](https://mlsanity.com/paper/2110.09681/) (2021)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks](https://mlsanity.com/paper/2212.02374/) (2022)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [GKAN: Graph Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.06470/) (2024)
- [Structural Adapters in Pretrained Language Models for AMR-to-text Generation](https://mlsanity.com/paper/2103.09120/) (2021)
- [Learning Multi-Agent Communication from Graph Modeling Perspective](https://mlsanity.com/paper/2405.08550/) (2024)
- [Relational Attention: Generalizing Transformers for Graph-Structured Tasks](https://mlsanity.com/paper/2210.05062/) (2022)
- [Rewiring with Positional Encodings for Graph Neural Networks](https://mlsanity.com/paper/2201.12674/) (2022)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [Robot Cooking with Stir-fry: Bimanual Non-prehensile Manipulation of Semi-fluid Objects](https://mlsanity.com/paper/2205.05960/) (2022)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [Over-Squashing in Graph Neural Networks: A Comprehensive survey](https://mlsanity.com/paper/2308.15568/) (2023)
- [Label Information Enhanced Fraud Detection against Low Homophily in Graphs](https://mlsanity.com/paper/2302.10407/) (2023)
- [A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers](https://mlsanity.com/paper/2306.02051/) (2023)
- [RNTrajRec: Road Network Enhanced Trajectory Recovery with Spatial-Temporal Transformer](https://mlsanity.com/paper/2211.13234/) (2022)
- [Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer](https://mlsanity.com/paper/2307.04895/) (2023)
- [Graph Decision Transformer](https://mlsanity.com/paper/2303.03747/) (2023)
- [Test-Time Training for Graph Neural Networks](https://mlsanity.com/paper/2210.08813/) (2022)
- [GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation](https://mlsanity.com/paper/2204.06674/) (2022)
- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [Unifying Structured Data as Graph for Data-to-Text Pre-Training](https://mlsanity.com/paper/2401.01183/) (2024)
- [Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning](https://mlsanity.com/paper/2501.01124/) (2025)
- [BERT-GT: Cross-sentence n-ary relation extraction with BERT and Graph Transformer](https://mlsanity.com/paper/2101.04158/) (2021)
- [Graphine: A Dataset for Graph-aware Terminology Definition Generation](https://mlsanity.com/paper/2109.04018/) (2021)

## Abstract (arXiv)

The dominant graph-to-sequence transduction models employ graph neural networks for graph representation learning, where the structural information is reflected by the receptive field of neurons. Unlike graph neural networks that restrict the information exchange between immediate neighborhood, we propose a new model, known as Graph Transformer, that uses explicit relation encoding and allows direct communication between two distant nodes. It provides a more efficient way for global graph structure modeling. Experiments on the applications of text generation from Abstract Meaning Representation (AMR) and syntax-based neural machine translation show the superiority of our proposed model. Specifically, our model achieves 27.4 BLEU on LDC2015E86 and 29.7 BLEU on LDC2017T10 for AMR-to-text generation, outperforming the state-of-the-art results by up to 2.2 points. On the syntax-based translation tasks, our model establishes new single-model state-of-the-art BLEU scores, 21.3 for English-to-German and 14.1 for English-to-Czech, improving over the existing best results, including ensembles, by over 1 BLEU.