Data to 7 Oct 2026

Graph Transformer Networks

arXiv 1911.06455 ↗First posted 6 Nov 2019cs.LG cs.SI stat.MLcited by 520

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have been widely used in representation learning on graphs and achieved state-of-the-art performance in tasks such as node classification and link prediction. However, most existing GNNs are designed to learn node representations on the fixed and homogeneous graphs. The limitations especially become problematic when learning representations on a misspecified graph or a heterogeneous graph that consists of various types of nodes and edges. In this paper, we propose Graph Transformer Networks (GTNs) that are capable of generating new graph structures, which involve identifying useful connections between unconnected nodes on the original graph, while learning effective node representation on the new graphs in an end-to-end fashion. Graph Transformer layer, a core layer of GTNs, learns a soft selection of edge types and composite relations for generating useful multi-hop connections so-called meta-paths. Our experiments show that GTNs learn new graph structures, based on data and tasks without domain knowledge, and yield powerful node representation via convolution on the new graphs. Without domain-specific graph preprocessing, GTNs achieved the best performance in all three benchmark node classification tasks against the state-of-the-art methods that require pre-defined meta-paths from domain knowledge.

Key facts. Cited by 520 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 67; 2022: 74; 2023: 101; 2024: 117; 2025: 100; 2026: 61.

520 papers.

Sort:

By area:

2022Transformers learn in-context by gradient descentcited by 714
2021Rethinking Graph Transformers with Spectral Attentioncited by 468
2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
2024LLaGA: Large Language and Graph Assistantcited by 177
2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
2023All in One: Multi-task Prompting for Graph Neural Networkscited by 147
2021GraphiT: Encoding Graph Structure in Transformerscited by 146
2024Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spacescited by 132
2021Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learningcited by 130
2021You are AllSet: A Multiset Function Framework for Hypergraph Neural Networkscited by 124
2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113

38 more

2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 110
2021Deep Generative Models in Engineering Design: A Reviewcited by 103
2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
2022Multi-Objective GFlowNetscited by 100
2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 84
2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
2021Graph Representation Learning in Biomedicinecited by 79
2022Deep Clustering: A Comprehensive Surveycited by 78
2023Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPTcited by 78
2023Temporal Data Meets LLM -- Explainable Financial Time Series Forecastingcited by 77
2024OpenGraph: Towards Open Graph Foundation Modelscited by 75
2024A Survey of Large Language Models for Graphscited by 71
2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 70
2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 65
2022Video Graph Transformer for Video Question Answeringcited by 63
2022On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networkscited by 63
2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 61
2021The Dawn of Quantum Natural Language Processingcited by 60
2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 60
2022Interpretable Graph Neural Networks for Connectome-Based Brain Disorder Analysiscited by 58
2021Anomaly Detection in Dynamic Graphs via Transformercited by 57
2023Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptionscited by 55
2021Knowledge-Preserving Incremental Social Event Detection via Heterogeneous GNNscited by 51
2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
2023GSLB: The Graph Structure Learning Benchmarkcited by 49
2023Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learningcited by 47
2022Shortest Path Networks for Graph Property Predictioncited by 46
2023MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphscited by 40
2022Mask and Reason: Pre-Training Knowledge Graph Transformers for Complex Logical Queriescited by 39
2023Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphscited by 39
2024RGFN: Synthesizable Molecular Generation Using GFlowNetscited by 39
2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 38
2021Boosting the Speed of Entity Alignment 10*: Dual Attention Matching Network with Normalized Hard Sample Miningcited by 37
2021The Threat of Offensive AI to Organizationscited by 37
2021Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methodscited by 36
2024On the Scalability of GNNs for Molecular Graphscited by 36
2022Rewiring with Positional Encodings for Graph Neural Networkscited by 35

50 most cited of 520 All 520 (JSON)