# Are Graph Attention Networks Able to Model Structural Information?

arXiv: https://arxiv.org/abs/2505.21288 · HTML: https://mlsanity.com/paper/2505.21288/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 22 of 22)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Self-Attention Graph Pooling](https://mlsanity.com/paper/1904.08082/) (2019)
- [struc2vec: Learning Node Representations from Structural Identity](https://mlsanity.com/paper/1704.03165/) (2017)
- [Graph Neural Networks for Graphs with Heterophily: A Survey](https://mlsanity.com/paper/2202.07082/) (2022)
- [ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations](https://mlsanity.com/paper/1911.07979/) (2019)
- [Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature](https://mlsanity.com/paper/2211.15779/) (2022)
- [Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels](https://mlsanity.com/paper/2105.04522/) (2021)
- [Edge Contraction Pooling for Graph Neural Networks](https://mlsanity.com/paper/1905.10990/) (2019)
- [KerGNNs: Interpretable Graph Neural Networks with Graph Kernels](https://mlsanity.com/paper/2201.00491/) (2022)
- [Anonymous Walk Embeddings](https://mlsanity.com/paper/1805.11921/) (2018)
- [RAW-GNN: RAndom Walk Aggregation based Graph Neural Network](https://mlsanity.com/paper/2206.13953/) (2022)
- [Learning Long Range Dependencies on Graphs via Random Walks](https://mlsanity.com/paper/2406.03386/) (2024)
- [Graph Kernel Neural Networks](https://mlsanity.com/paper/2112.07436/) (2021)
- [A Quantum-inspired Similarity Measure for the Analysis of Complete Weighted Graphs](https://mlsanity.com/paper/1904.13239/) (2019)
- [Rethinking Kernel Methods for Node Representation Learning on Graphs](https://mlsanity.com/paper/1910.02548/) (2019)
- [Representation Learning on Heterostructures via Heterogeneous Anonymous Walks](https://mlsanity.com/paper/2201.06972/) (2022)

## Abstract (arXiv)

Graph Attention Networks (GATs) have emerged as powerful models for learning expressive representations from such data by adaptively weighting neighboring nodes through attention mechanisms. However, most existing approaches primarily rely on node attributes and direct neighborhood connections, often overlooking rich structural patterns that capture higher-order topological information crucial for many real-world datasets. In this work, we present the Graph Structure Attention Network (GSAT), a novel extension of GAT that jointly integrates attribute-based and structure-based representations for more effective graph learning. GSAT incorporates structural features derived from anonymous random walks (ARWs) and graph kernels to encode local topological information, enabling attention mechanisms to adapt based on the underlying graph structure. This design enhances the model's ability to discern meaningful relational dependencies within complex data. Comprehensive experiments on standard graph classification and regression benchmarks demonstrate that GSAT achieves consistent improvements over state-of-the-art graph learning methods, highlighting the value of incorporating structural context for representation learning on graphs.