# TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems

arXiv: https://arxiv.org/abs/2308.14355 · HTML: https://mlsanity.com/paper/2308.14355/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 58 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Adaptive Universal Generalized PageRank Graph Neural Network](https://mlsanity.com/paper/2006.07988/) (2020)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [A Note on Over-Smoothing for Graph Neural Networks](https://mlsanity.com/paper/2006.13318/) (2020)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://mlsanity.com/paper/2210.14709/) (2022)
- [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://mlsanity.com/paper/2105.02605/) (2021)
- [Neural Collaborative Filtering vs. Matrix Factorization Revisited](https://mlsanity.com/paper/2005.09683/) (2020)
- [Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach](https://mlsanity.com/paper/2001.10167/) (2020)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [STAN: Spatio-Temporal Attention Network for Next Location Recommendation](https://mlsanity.com/paper/2102.04095/) (2021)
- [Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation](https://mlsanity.com/paper/2101.06448/) (2021)
- [Breaking the Limits of Message Passing Graph Neural Networks](https://mlsanity.com/paper/2106.04319/) (2021)
- [Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer](https://mlsanity.com/paper/2108.06625/) (2021)
- [Knowledge-aware Coupled Graph Neural Network for Social Recommendation](https://mlsanity.com/paper/2110.03987/) (2021)
- [Efficiently Leveraging Multi-level User Intent for Session-based Recommendation via Atten-Mixer Network](https://mlsanity.com/paper/2206.12781/) (2022)

## Built on by (most cited first, 11 of 11)

- [A Survey of Graph Transformers: Architectures, Theories and Applications](https://mlsanity.com/paper/2502.16533/) (2025)
- [All-in-One: Heterogeneous Interaction Modeling for Cold-Start Rating Prediction](https://mlsanity.com/paper/2403.17740/) (2024)
- [Graph Learning Metallic Glass Discovery from Wikipedia](https://mlsanity.com/paper/2507.19536/) (2025)
- [Question-Adaptive Graph Learning for Multi-hop Retrieval Augmented Generation](https://mlsanity.com/paper/2510.11541/) (2025)
- [GVTNet: Graph Vision Transformer For Face Super-Resolution](https://mlsanity.com/paper/2502.12570/) (2025)
- [Multi-Perspective Attention Mechanism for Bias-Aware Sequential Recommendation](https://mlsanity.com/paper/2504.05323/) (2025)
- [Panning for Gold: Expanding Domain-Specific Knowledge Graphs with General Knowledge](https://mlsanity.com/paper/2601.10485/) (2026)
- [BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning](https://mlsanity.com/paper/2604.06336/) (2026)
- [Dynamic Graph with Similarity-Aware Attention Graph Neural Network for Recommender Systems](https://mlsanity.com/paper/2605.05238/) (2026)
- [Boundary Embedding Shaping with Adaptive Contrastive Learning for Graph Structural Disentanglement](https://mlsanity.com/paper/2606.20283/) (2026)
- [X-LogSMask: Expand Transformer for Graph-Structured Data](https://mlsanity.com/paper/2607.01553/) (2026)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have emerged as promising solutions for collaborative filtering (CF) through the modeling of user-item interaction graphs. The nucleus of existing GNN-based recommender systems involves recursive message passing along user-item interaction edges to refine encoded embeddings. Despite their demonstrated effectiveness, current GNN-based methods encounter challenges of limited receptive fields and the presence of noisy "interest-irrelevant" connections. In contrast, Transformer-based methods excel in aggregating information adaptively and globally. Nevertheless, their application to large-scale interaction graphs is hindered by inherent complexities and challenges in capturing intricate, entangled structural information. In this paper, we propose TransGNN, a novel model that integrates Transformer and GNN layers in an alternating fashion to mutually enhance their capabilities. Specifically, TransGNN leverages Transformer layers to broaden the receptive field and disentangle information aggregation from edges, which aggregates information from more relevant nodes, thereby enhancing the message passing of GNNs. Additionally, to capture graph structure information effectively, positional encoding is meticulously designed and integrated into GNN layers to encode such structural knowledge into node attributes, thus enhancing the Transformer's performance on graphs. Efficiency considerations are also alleviated by proposing the sampling of the most relevant nodes for the Transformer, along with two efficient sample update strategies to reduce complexity. Furthermore, theoretical analysis demonstrates that TransGNN offers increased expressiveness compared to GNNs, with only a marginal increase in linear complexity. Extensive experiments on five public datasets validate the effectiveness and efficiency of TransGNN.