# Universal Graph Transformer Self-Attention Networks

arXiv: https://arxiv.org/abs/1909.11855 · HTML: https://mlsanity.com/paper/1909.11855/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2019.

## Built on by (most cited first, 30 of 41)

- [Pure Transformers are Powerful Graph Learners](https://mlsanity.com/paper/2207.02505/) (2022)
- [Transformer for Graphs: An Overview from Architecture Perspective](https://mlsanity.com/paper/2202.08455/) (2022)
- [Graph Mamba: Towards Learning on Graphs with State Space Models](https://mlsanity.com/paper/2402.08678/) (2024)
- [Energy Transformer](https://mlsanity.com/paper/2302.07253/) (2023)
- [FedGTA: Topology-aware Averaging for Federated Graph Learning](https://mlsanity.com/paper/2401.11755/) (2024)
- [ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection](https://mlsanity.com/paper/2110.07317/) (2021)
- [Graph Transformers: A Survey](https://mlsanity.com/paper/2407.09777/) (2024)
- [Node Co-occurrence based Graph Neural Networks for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2104.07396/) (2021)
- [Joint Graph Learning and Matching for Semantic Feature Correspondence](https://mlsanity.com/paper/2109.00240/) (2021)
- [PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations](https://mlsanity.com/paper/2211.14425/) (2022)
- [A Survey on Graph Classification and Link Prediction based on GNN](https://mlsanity.com/paper/2307.00865/) (2023)
- [A Survey on Structure-Preserving Graph Transformers](https://mlsanity.com/paper/2401.16176/) (2024)
- [Union Subgraph Neural Networks](https://mlsanity.com/paper/2305.15747/) (2023)
- [An Interpretable Ensemble of Graph and Language Models for Improving Search Relevance in E-Commerce](https://mlsanity.com/paper/2403.00923/) (2024)
- [AGFormer: Efficient Graph Representation with Anchor-Graph Transformer](https://mlsanity.com/paper/2305.07521/) (2023)
- [Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour](https://mlsanity.com/paper/2208.03819/) (2022)
- [HGCN(O): A Self-Tuning GCN HyperModel Toolkit for Outcome Prediction in Event-Sequence Data](https://mlsanity.com/paper/2507.22524/) (2025)
- [EGC2: Enhanced Graph Classification with Easy Graph Compression](https://mlsanity.com/paper/2107.07737/) (2021)
- [Two-view Graph Neural Networks for Knowledge Graph Completion](https://mlsanity.com/paper/2112.09231/) (2021)
- [A Class-Aware Representation Refinement Framework for Graph Classification](https://mlsanity.com/paper/2209.00936/) (2022)
- [Tensor-Fused Multi-View Graph Contrastive Learning](https://mlsanity.com/paper/2410.15247/) (2024)
- [Saliency-Aware Regularized Graph Neural Network](https://mlsanity.com/paper/2401.00755/) (2024)
- [Graph Inverse Style Transfer for Counterfactual Explainability](https://mlsanity.com/paper/2505.17542/) (2025)
- [AutoHEnsGNN: Winning Solution to AutoGraph Challenge for KDD Cup 2020](https://mlsanity.com/paper/2111.12952/) (2021)
- [Inducing Gaussian Process Networks](https://mlsanity.com/paper/2204.09889/) (2022)
- [A Graph Transformer-Driven Approach for Network Robustness Learning](https://mlsanity.com/paper/2306.06913/) (2023)
- [Multiscale Hodge Scattering Networks for Data Analysis](https://mlsanity.com/paper/2311.10270/) (2023)
- [Accelerating Flood Warnings by 10 Hours: The Power of River Network Topology in AI-enhanced Flood Forecasting](https://mlsanity.com/paper/2410.05536/) (2024)
- [KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks](https://mlsanity.com/paper/2501.13456/) (2025)
- [Divergent Paths: Separating Homophilic and Heterophilic Learning for Enhanced Graph-level Representations](https://mlsanity.com/paper/2504.05344/) (2025)

## Abstract (arXiv)

We introduce a transformer-based GNN model, named UGformer, to learn graph representations. In particular, we present two UGformer variants, wherein the first variant (publicized in September 2019) is to leverage the transformer on a set of sampled neighbors for each input node, while the second (publicized in May 2021) is to leverage the transformer on all input nodes. Experimental results demonstrate that the first UGformer variant achieves state-of-the-art accuracies on benchmark datasets for graph classification in both inductive setting and unsupervised transductive setting; and the second UGformer variant obtains state-of-the-art accuracies for inductive text classification. The code is available at: urlhttps://github.com/daiquocnguyen/Graph-Transformer.