# Graph Representation Learning via Hard and Channel-Wise Attention Networks

arXiv: https://arxiv.org/abs/1907.04652 · HTML: https://mlsanity.com/paper/1907.04652/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 18 of 18)

- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs](https://mlsanity.com/paper/2206.11990/) (2022)
- [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://mlsanity.com/paper/2102.10757/) (2021)
- [How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision](https://mlsanity.com/paper/2204.04879/) (2022)
- [A Comprehensive Survey on Deep Graph Representation Learning](https://mlsanity.com/paper/2304.05055/) (2023)
- [A Survey on Graph Structure Learning: Progress and Opportunities](https://mlsanity.com/paper/2103.03036/) (2021)
- [RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting](https://mlsanity.com/paper/2108.01316/) (2021)
- [TraverseNet: Unifying Space and Time in Message Passing for Traffic Forecasting](https://mlsanity.com/paper/2109.02474/) (2021)
- [Collaborative Graph Neural Networks for Attributed Network Embedding](https://mlsanity.com/paper/2307.11981/) (2023)
- [A Variational Edge Partition Model for Supervised Graph Representation Learning](https://mlsanity.com/paper/2202.03233/) (2022)
- [Not All Neighbors Are Worth Attending to: Graph Selective Attention Networks for Semi-supervised Learning](https://mlsanity.com/paper/2210.07715/) (2022)
- [Node2Seq: Towards Trainable Convolutions in Graph Neural Networks](https://mlsanity.com/paper/2101.01849/) (2021)
- [Group Contrastive Self-Supervised Learning on Graphs](https://mlsanity.com/paper/2107.09787/) (2021)
- [Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention](https://mlsanity.com/paper/2305.13115/) (2023)
- [Revisiting 2D Convolutional Neural Networks for Graph-based Applications](https://mlsanity.com/paper/2105.11016/) (2021)
- [Attention-based graph neural networks: a survey](https://mlsanity.com/paper/2605.08679/) (2026)
- [Geometric Flow enhanced Graph Coarsening](https://mlsanity.com/paper/2609.14962/) (2026)
- [CoRGi: Content-Rich Graph Neural Networks with Attention](https://mlsanity.com/paper/2110.04866/) (2021)

## Abstract (arXiv)

Attention operators have been widely applied in various fields, including computer vision, natural language processing, and network embedding learning. Attention operators on graph data enables learnable weights when aggregating information from neighboring nodes. However, graph attention operators (GAOs) consume excessive computational resources, preventing their applications on large graphs. In addition, GAOs belong to the family of soft attention, instead of hard attention, which has been shown to yield better performance. In this work, we propose novel hard graph attention operator (hGAO) and channel-wise graph attention operator (cGAO). hGAO uses the hard attention mechanism by attending to only important nodes. Compared to GAO, hGAO improves performance and saves computational cost by only attending to important nodes. To further reduce the requirements on computational resources, we propose the cGAO that performs attention operations along channels. cGAO avoids the dependency on the adjacency matrix, leading to dramatic reductions in computational resource requirements. Experimental results demonstrate that our proposed deep models with the new operators achieve consistently better performance. Comparison results also indicates that hGAO achieves significantly better performance than GAO on both node and graph embedding tasks. Efficiency comparison shows that our cGAO leads to dramatic savings in computational resources, making them applicable to large graphs.