# Higher-Order Attribute-Enhancing Heterogeneous Graph Neural Networks

arXiv: https://arxiv.org/abs/2104.07892 · HTML: https://mlsanity.com/paper/2104.07892/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](https://mlsanity.com/paper/1509.09292/) (2015)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding](https://mlsanity.com/paper/2002.01680/) (2020)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Diffusion-Convolutional Neural Networks](https://mlsanity.com/paper/1511.02136/) (2015)
- [Deep Convolutional Networks on Graph-Structured Data](https://mlsanity.com/paper/1506.05163/) (2015)
- [A Survey on Network Embedding](https://mlsanity.com/paper/1711.08752/) (2017)
- [A Survey of Heterogeneous Information Network Analysis](https://mlsanity.com/paper/1511.04854/) (2015)
- [Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification](https://mlsanity.com/paper/1906.04898/) (2019)
- [Streaming Social Event Detection and Evolution Discovery in Heterogeneous Information Networks](https://mlsanity.com/paper/2104.00853/) (2021)
- [Fine-grained Event Categorization with Heterogeneous Graph Convolutional Networks](https://mlsanity.com/paper/1906.04580/) (2019)
- [SHINE: Signed Heterogeneous Information Network Embedding for Sentiment Link Prediction](https://mlsanity.com/paper/1712.00732/) (2017)
- [Task-Guided and Path-Augmented Heterogeneous Network Embedding for Author Identification](https://mlsanity.com/paper/1612.02814/) (2016)

## Built on by (most cited first, 22 of 22)

- [OOD-GNN: Out-of-Distribution Generalized Graph Neural Network](https://mlsanity.com/paper/2112.03806/) (2021)
- [Explaining the Explainers in Graph Neural Networks: a Comparative Study](https://mlsanity.com/paper/2210.15304/) (2022)
- [Higher-order Clustering and Pooling for Graph Neural Networks](https://mlsanity.com/paper/2209.03473/) (2022)
- [ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network](https://mlsanity.com/paper/2110.07888/) (2021)
- [Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining](https://mlsanity.com/paper/2403.04780/) (2024)
- [Efficient Heterogeneous Graph Learning via Random Projection](https://mlsanity.com/paper/2310.14481/) (2023)
- [Individual and Structural Graph Information Bottlenecks for Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.15902/) (2023)
- [Projective Ranking-based GNN Evasion Attacks](https://mlsanity.com/paper/2202.12993/) (2022)
- [Large Language Model Meets Graph Neural Network in Knowledge Distillation](https://mlsanity.com/paper/2402.05894/) (2024)
- [Boosting-GNN: Boosting Algorithm for Graph Networks on Imbalanced Node Classification](https://mlsanity.com/paper/2105.11625/) (2021)
- [RARE: Robust Masked Graph Autoencoder](https://mlsanity.com/paper/2304.01507/) (2023)
- [Graph Convolutional Neural Networks with Diverse Negative Samples via Decomposed Determinant Point Processes](https://mlsanity.com/paper/2212.02055/) (2022)
- [Symmetric Graph Contrastive Learning against Noisy Views for Recommendation](https://mlsanity.com/paper/2408.02691/) (2024)
- [Towards Fine-Grained Explainability for Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2312.15237/) (2023)
- [Long-range Meta-path Search on Large-scale Heterogeneous Graphs](https://mlsanity.com/paper/2307.08430/) (2023)
- [Time-aware Hyperbolic Graph Attention Network for Session-based Recommendation](https://mlsanity.com/paper/2301.03780/) (2023)
- [SCGG: A Deep Structure-Conditioned Graph Generative Model](https://mlsanity.com/paper/2209.09681/) (2022)
- [KGIF: Optimizing Relation-Aware Recommendations with Knowledge Graph Information Fusion](https://mlsanity.com/paper/2501.04161/) (2025)
- [ScaleGNN: Towards Scalable Graph Neural Networks via Adaptive High-order Neighboring Feature Fusion](https://mlsanity.com/paper/2504.15920/) (2025)
- [Heterogeneous Temporal Hypergraph Neural Network](https://mlsanity.com/paper/2506.17312/) (2025)
- [Efficient Knowledge Tracing Leveraging Higher-Order Information in Integrated Graphs](https://mlsanity.com/paper/2507.18668/) (2025)
- [Graph Integrated Transformers for Community Detection in Social Networks](https://mlsanity.com/paper/2601.04367/) (2026)

## Abstract (arXiv)

Graph neural networks (GNNs) have been widely used in deep learning on graphs. They can learn effective node representations that achieve superior performances in graph analysis tasks such as node classification and node clustering. However, most methods ignore the heterogeneity in real-world graphs. Methods designed for heterogeneous graphs, on the other hand, fail to learn complex semantic representations because they only use meta-paths instead of meta-graphs. Furthermore, they cannot fully capture the content-based correlations between nodes, as they either do not use the self-attention mechanism or only use it to consider the immediate neighbors of each node, ignoring the higher-order neighbors. We propose a novel Higher-order Attribute-Enhancing (HAE) framework that enhances node embedding in a layer-by-layer manner. Under the HAE framework, we propose a Higher-order Attribute-Enhancing Graph Neural Network (HAEGNN) for heterogeneous network representation learning. HAEGNN simultaneously incorporates meta-paths and meta-graphs for rich, heterogeneous semantics, and leverages the self-attention mechanism to explore content-based nodes interactions. The unique higher-order architecture of HAEGNN allows examining the first-order as well as higher-order neighborhoods. Moreover, HAEGNN shows good explainability as it learns the importances of different meta-paths and meta-graphs. HAEGNN is also memory-efficient, for it avoids per meta-path based matrix calculation. Experimental results not only show HAEGNN superior performance against the state-of-the-art methods in node classification, node clustering, and visualization, but also demonstrate its superiorities in terms of memory efficiency and explainability.