# Attention-based Graph Neural Network for Semi-supervised Learning

arXiv: https://arxiv.org/abs/1803.03735 · HTML: https://mlsanity.com/paper/1803.03735/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2018.

## Built on by (most cited first, 30 of 101)

- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Graph neural networks for materials science and chemistry](https://mlsanity.com/paper/2208.09481/) (2022)
- [LLaGA: Large Language and Graph Assistant](https://mlsanity.com/paper/2402.08170/) (2024)
- [Causal Attention for Interpretable and Generalizable Graph Classification](https://mlsanity.com/paper/2112.15089/) (2021)
- [How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision](https://mlsanity.com/paper/2204.04879/) (2022)
- [Graph-based Semi-supervised Learning: A Comprehensive Review](https://mlsanity.com/paper/2102.13303/) (2021)
- [A General Survey on Attention Mechanisms in Deep Learning](https://mlsanity.com/paper/2203.14263/) (2022)
- [Learning to Reconstruct Missing Data from Spatiotemporal Graphs with Sparse Observations](https://mlsanity.com/paper/2205.13479/) (2022)
- [Graph Neural Networks Inspired by Classical Iterative Algorithms](https://mlsanity.com/paper/2103.06064/) (2021)
- [Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth](https://mlsanity.com/paper/2105.04550/) (2021)
- [Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2106.04054/) (2021)
- [Boost then Convolve: Gradient Boosting Meets Graph Neural Networks](https://mlsanity.com/paper/2101.08543/) (2021)
- [p-Laplacian Based Graph Neural Networks](https://mlsanity.com/paper/2111.07337/) (2021)
- [Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift](https://mlsanity.com/paper/2211.02843/) (2022)
- [FedGL: Federated Graph Learning Framework with Global Self-Supervision](https://mlsanity.com/paper/2105.03170/) (2021)
- [Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/2205.09702/) (2022)
- [SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization](https://mlsanity.com/paper/2207.09179/) (2022)
- [Motif Prediction with Graph Neural Networks](https://mlsanity.com/paper/2106.00761/) (2021)
- [TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs](https://mlsanity.com/paper/2112.02052/) (2021)
- [How Expressive are Transformers in Spectral Domain for Graphs?](https://mlsanity.com/paper/2201.09332/) (2022)
- [Graph Neural Networks for Text Classification: A Survey](https://mlsanity.com/paper/2304.11534/) (2023)
- [Implicit vs Unfolded Graph Neural Networks](https://mlsanity.com/paper/2111.06592/) (2021)
- [U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds](https://mlsanity.com/paper/2308.06383/) (2023)
- [You too Brutus! Trapping Hateful Users in Social Media: Challenges, Solutions & Insights](https://mlsanity.com/paper/2108.00524/) (2021)
- [Neural Graph Databases](https://mlsanity.com/paper/2209.09732/) (2022)
- [Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again](https://mlsanity.com/paper/2210.08122/) (2022)
- [HOT: Higher-Order Dynamic Graph Representation Learning with Efficient Transformers](https://mlsanity.com/paper/2311.18526/) (2023)
- [Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features](https://mlsanity.com/paper/2110.13413/) (2021)
- [FL-AGCNS: Federated Learning Framework for Automatic Graph Convolutional Network Search](https://mlsanity.com/paper/2104.04141/) (2021)
- [Graph Decoupling Attention Markov Networks for Semi-supervised Graph Node Classification](https://mlsanity.com/paper/2104.13718/) (2021)

## Abstract (arXiv)

Recently popularized graph neural networks achieve the state-of-the-art accuracy on a number of standard benchmark datasets for graph-based semi-supervised learning, improving significantly over existing approaches. These architectures alternate between a propagation layer that aggregates the hidden states of the local neighborhood and a fully-connected layer. Perhaps surprisingly, we show that a linear model, that removes all the intermediate fully-connected layers, is still able to achieve a performance comparable to the state-of-the-art models. This significantly reduces the number of parameters, which is critical for semi-supervised learning where number of labeled examples are small. This in turn allows a room for designing more innovative propagation layers. Based on this insight, we propose a novel graph neural network that removes all the intermediate fully-connected layers, and replaces the propagation layers with attention mechanisms that respect the structure of the graph. The attention mechanism allows us to learn a dynamic and adaptive local summary of the neighborhood to achieve more accurate predictions. In a number of experiments on benchmark citation networks datasets, we demonstrate that our approach outperforms competing methods. By examining the attention weights among neighbors, we show that our model provides some interesting insights on how neighbors influence each other.