# Graph Transformer Networks: Learning Meta-path Graphs to Improve GNNs

arXiv: https://arxiv.org/abs/2106.06218 · HTML: https://mlsanity.com/paper/2106.06218/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing](https://mlsanity.com/paper/1905.00067/) (2019)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Composition-based Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/1911.03082/) (2019)
- [Graph Structure Learning for Robust Graph Neural Networks](https://mlsanity.com/paper/2005.10203/) (2020)
- [Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation](https://mlsanity.com/paper/1806.02473/) (2018)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Towards Deeper Graph Neural Networks](https://mlsanity.com/paper/2007.09296/) (2020)
- [Inductive Relation Prediction by Subgraph Reasoning](https://mlsanity.com/paper/1911.06962/) (2019)
- [Efficient Graph Generation with Graph Recurrent Attention Networks](https://mlsanity.com/paper/1910.00760/) (2019)

## Built on by (most cited first, 22 of 22)

- [Generative Flows on Synthetic Pathway for Drug Design](https://mlsanity.com/paper/2410.04542/) (2024)
- [Meta-Path Learning for Multi-relational Graph Neural Networks](https://mlsanity.com/paper/2309.17113/) (2023)
- [A Self-Explainable Heterogeneous GNN for Relational Deep Learning](https://mlsanity.com/paper/2412.00521/) (2024)
- [Compositional Flows for 3D Molecule and Synthesis Pathway Co-design](https://mlsanity.com/paper/2504.08051/) (2025)
- [Task-Oriented GNNs Training on Large Knowledge Graphs for Accurate and Efficient Modeling](https://mlsanity.com/paper/2403.05752/) (2024)
- [A Neuro-Symbolic Framework for Answering Graph Pattern Queries in Knowledge Graphs](https://mlsanity.com/paper/2310.04598/) (2023)
- [Structured Click Control in Transformer-based Interactive Segmentation](https://mlsanity.com/paper/2405.04009/) (2024)
- [Leveraging Medical Foundation Model Features in Graph Neural Network-Based Retrieval of Breast Histopathology Images](https://mlsanity.com/paper/2405.04211/) (2024)
- [Decision-focused Graph Neural Networks for Combinatorial Optimization](https://mlsanity.com/paper/2406.03647/) (2024)
- [Flexible ViG: Learning the Self-Saliency for Flexible Object Recognition](https://mlsanity.com/paper/2406.18585/) (2024)
- [HHGT: Hierarchical Heterogeneous Graph Transformer for Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2407.13158/) (2024)
- [Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs](https://mlsanity.com/paper/2307.12179/) (2023)
- [Modeling Balanced Explicit and Implicit Relations with Contrastive Learning for Knowledge Concept Recommendation in MOOCs](https://mlsanity.com/paper/2402.08256/) (2024)
- [Attention-based graph neural networks: a survey](https://mlsanity.com/paper/2605.08679/) (2026)
- [Meta-node: A Concise Approach to Effectively Learn Complex Relationships in Heterogeneous Graphs](https://mlsanity.com/paper/2210.14480/) (2022)
- [Unifying gradient regularization for Heterogeneous Graph Neural Networks](https://mlsanity.com/paper/2305.15811/) (2023)
- [Inferring Pluggable Types with Machine Learning](https://mlsanity.com/paper/2406.15676/) (2024)
- [An Extended Kalman Filter Integrated Latent Feature Model on Dynamic Weighted Directed Graphs](https://mlsanity.com/paper/2407.21376/) (2024)
- [Corporate Fraud Detection in Rich-yet-Noisy Financial Graph](https://mlsanity.com/paper/2502.19305/) (2025)
- [FoGE: Fock Space inspired encoding for graph prompting](https://mlsanity.com/paper/2507.02937/) (2025)
- [GCL-GCN: Graphormer and Contrastive Learning Enhanced Attributed Graph Clustering Network](https://mlsanity.com/paper/2507.19095/) (2025)
- [NeuroPathNet: Dynamic Path Trajectory Learning for Brain Functional Connectivity Analysis](https://mlsanity.com/paper/2510.24025/) (2025)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have been widely applied to various fields due to their powerful representations of graph-structured data. Despite the success of GNNs, most existing GNNs are designed to learn node representations on the fixed and homogeneous graphs. The limitations especially become problematic when learning representations on a misspecified graph or a heterogeneous graph that consists of various types of nodes and edges. To address this limitations, we propose Graph Transformer Networks (GTNs) that are capable of generating new graph structures, which preclude noisy connections and include useful connections (e.g., meta-paths) for tasks, while learning effective node representations on the new graphs in an end-to-end fashion. We further propose enhanced version of GTNs, Fast Graph Transformer Networks (FastGTNs), that improve scalability of graph transformations. Compared to GTNs, FastGTNs are 230x faster and use 100x less memory while allowing the identical graph transformations as GTNs. In addition, we extend graph transformations to the semantic proximity of nodes allowing non-local operations beyond meta-paths. Extensive experiments on both homogeneous graphs and heterogeneous graphs show that GTNs and FastGTNs with non-local operations achieve the state-of-the-art performance for node classification tasks. The code is available: https://github.com/seongjunyun/Graph_Transformer_Networks