# SlotGAT: Slot-based Message Passing for Heterogeneous Graph Neural Network

arXiv: https://arxiv.org/abs/2405.01927 · HTML: https://mlsanity.com/paper/2405.01927/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 29 of 29)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding](https://mlsanity.com/paper/2002.01680/) (2020)
- [GPT-GNN: Generative Pre-Training of Graph Neural Networks](https://mlsanity.com/paper/2006.15437/) (2020)
- [Towards Deeper Graph Neural Networks](https://mlsanity.com/paper/2007.09296/) (2020)
- [Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks](https://mlsanity.com/paper/2112.14936/) (2021)
- [DeepInf: Social Influence Prediction with Deep Learning](https://mlsanity.com/paper/1807.05560/) (2018)
- [Knowledge Graph Convolutional Networks for Recommender Systems](https://mlsanity.com/paper/1904.12575/) (2019)
- [Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems](https://mlsanity.com/paper/1905.04413/) (2019)
- [Representation Learning for Attributed Multiplex Heterogeneous Network](https://mlsanity.com/paper/1905.01669/) (2019)
- [An Attention-based Graph Neural Network for Heterogeneous Structural Learning](https://mlsanity.com/paper/1912.10832/) (2019)
- [DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation](https://mlsanity.com/paper/2106.10879/) (2021)
- [Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2202.09177/) (2022)

## Built on by (most cited first, 11 of 11)

- [Road Network Representation Learning with the Third Law of Geography](https://mlsanity.com/paper/2406.04038/) (2024)
- [Heterogeneous Sheaf Neural Networks](https://mlsanity.com/paper/2409.08036/) (2024)
- [Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendation](https://mlsanity.com/paper/2504.02458/) (2025)
- [TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation](https://mlsanity.com/paper/2412.11180/) (2024)
- [GLProtein: Global-and-Local Structure Aware Protein Representation Learning](https://mlsanity.com/paper/2506.06294/) (2025)
- [Simple yet Effective Graph Distillation via Clustering](https://mlsanity.com/paper/2505.20807/) (2025)
- [GegenNet: Spectral Convolutional Neural Networks for Link Sign Prediction in Signed Bipartite Graphs](https://mlsanity.com/paper/2508.19907/) (2025)
- [Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach](https://mlsanity.com/paper/2412.08038/) (2024)
- [Are Heterogeneous Graph Neural Networks Truly Effective for Node Classification? A Causal Perspective](https://mlsanity.com/paper/2510.05750/) (2025)
- [Rethinking Message Passing Neural Networks with Diffusion Distance-guided Stress Majorization](https://mlsanity.com/paper/2511.19984/) (2025)
- [BDIP-Net: Dual-Interaction Graph Learning for Property Prediction of Bilayer Materials](https://mlsanity.com/paper/2608.14640/) (2026)

## Abstract (arXiv)

Heterogeneous graphs are ubiquitous to model complex data. There are urgent needs on powerful heterogeneous graph neural networks to effectively support important applications. We identify a potential semantic mixing issue in existing message passing processes, where the representations of the neighbors of a node v are forced to be transformed to the feature space of v for aggregation, though the neighbors are in different types. That is, the semantics in different node types are entangled together into node v's representation. To address the issue, we propose SlotGAT with separate message passing processes in slots, one for each node type, to maintain the representations in their own node-type feature spaces. Moreover, in a slot-based message passing layer, we design an attention mechanism for effective slot-wise message aggregation. Further, we develop a slot attention technique after the last layer of SlotGAT, to learn the importance of different slots in downstream tasks. Our analysis indicates that the slots in SlotGAT can preserve different semantics in various feature spaces. The superiority of SlotGAT is evaluated against 13 baselines on 6 datasets for node classification and link prediction. Our code is at https://github.com/scottjiao/SlotGATICML23/.