# GQWformer: A Quantum-based Transformer for Graph Representation Learning

arXiv: https://arxiv.org/abs/2412.02285 · HTML: https://mlsanity.com/paper/2412.02285/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 42 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 6 of 6)

- [TUDataset: A collection of benchmark datasets for learning with graphs](https://mlsanity.com/paper/2007.08663/) (2020)
- [GraphiT: Encoding Graph Structure in Transformers](https://mlsanity.com/paper/2106.05667/) (2021)
- [Walking Out of the Weisfeiler Leman Hierarchy: Graph Learning Beyond Message Passing](https://mlsanity.com/paper/2102.08786/) (2021)
- [Quantum Graph Learning: Frontiers and Outlook](https://mlsanity.com/paper/2302.00892/) (2023)
- [From Quantum Graph Computing to Quantum Graph Learning: A Survey](https://mlsanity.com/paper/2202.09506/) (2022)
- [Less is More: on the Over-Globalizing Problem in Graph Transformers](https://mlsanity.com/paper/2405.01102/) (2024)

## Built on by (most cited first, 1 of 1)

- [CTQWformer: A CTQW-based Transformer for Graph Classification](https://mlsanity.com/paper/2605.09486/) (2026)

## Abstract (arXiv)

Graph Transformers (GTs) have demonstrated significant advantages in graph representation learning through their global attention mechanisms. However, the self-attention mechanism in GTs tends to neglect the inductive biases inherent in graph structures, making it chanllenging to effectively capture essential structural information. To address this issue, we propose a novel approach that integrate graph inductive bias into self-attention mechanisms by leveraging quantum technology for structural encoding. In this paper, we introduce the Graph Quantum Walk Transformer (GQWformer), a groundbreaking GNN framework that utilizes quantum walks on attributed graphs to generate node quantum states. These quantum states encapsulate rich structural attributes and serve as inductive biases for the transformer, thereby enabling the generation of more meaningful attention scores. By subsequently incorporating a recurrent neural network, our design amplifies the model's ability to focus on both local and global information. We conducted comprehensive experiments across five publicly available datasets to evaluate the effectiveness of our model. These results clearly indicate that GQWformer outperforms existing state-of-the-art graph classification algorithms. These findings highlight the significant potential of integrating quantum computing methodologies with traditional GNNs to advance the field of graph representation learning, providing a promising direction for future research and applications.