# Sparse Graph Attention Networks

arXiv: https://arxiv.org/abs/1912.00552 · HTML: https://mlsanity.com/paper/1912.00552/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 30 of 41)

- [Data Augmentation for Deep Graph Learning: A Survey](https://mlsanity.com/paper/2202.08235/) (2022)
- [A Survey of Mamba](https://mlsanity.com/paper/2408.01129/) (2024)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation](https://mlsanity.com/paper/2208.09779/) (2022)
- [Uncertainty Quantification of Spatiotemporal Travel Demand with Probabilistic Graph Neural Networks](https://mlsanity.com/paper/2303.04040/) (2023)
- [FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learning](https://mlsanity.com/paper/2410.17555/) (2024)
- [Sparse Covariance Neural Networks](https://mlsanity.com/paper/2410.01669/) (2024)
- [AGS-GNN: Attribute-guided Sampling for Graph Neural Networks](https://mlsanity.com/paper/2405.15218/) (2024)
- [Causal-Based Supervision of Attention in Graph Neural Network: A Better and Simpler Choice towards Powerful Attention](https://mlsanity.com/paper/2305.13115/) (2023)
- [A Survey on Dropout Methods and Experimental Verification in Recommendation](https://mlsanity.com/paper/2204.02027/) (2022)
- [Triple Sparsification of Graph Convolutional Networks without Sacrificing the Accuracy](https://mlsanity.com/paper/2208.03559/) (2022)
- [Robust Node Representation Learning via Graph Variational Diffusion Networks](https://mlsanity.com/paper/2312.10903/) (2023)
- [Reliable Multi-Modal Object Re-Identification via Modality-Aware Graph Reasoning](https://mlsanity.com/paper/2504.14847/) (2025)
- [Respecting Time Series Properties Makes Deep Time Series Forecasting Perfect](https://mlsanity.com/paper/2207.10941/) (2022)
- [Spiking GATs: Learning Graph Attentions via Spiking Neural Network](https://mlsanity.com/paper/2209.13539/) (2022)
- [Causally-guided Regularization of Graph Attention Improves Generalizability](https://mlsanity.com/paper/2210.10946/) (2022)
- [Exploring Sparsity in Graph Transformers](https://mlsanity.com/paper/2312.05479/) (2023)
- [Graph-Based Physics-Guided Urban PM2.5 Air Quality Imputation with Constrained Monitoring Data](https://mlsanity.com/paper/2506.06917/) (2025)
- [EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification](https://mlsanity.com/paper/2311.12823/) (2023)
- [LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs](https://mlsanity.com/paper/2111.05694/) (2021)
- [Semantic Graph Neural Network with Multi-measure Learning for Semi-supervised Classification](https://mlsanity.com/paper/2212.01749/) (2022)
- [SFi-Former: Sparse Flow Induced Attention for Graph Transformer](https://mlsanity.com/paper/2504.20666/) (2025)
- [Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting](https://mlsanity.com/paper/2511.08888/) (2025)
- [Perspectives on Tsallis Statistics for Artificial Intelligence](https://mlsanity.com/paper/2608.01223/) (2026)
- [Mitigating Overfitting in Graph Neural Networks via Feature and Hyperplane Perturbation](https://mlsanity.com/paper/2211.15081/) (2022)
- [Interpretable System Identification and Long-term Prediction on Time-Series Data](https://mlsanity.com/paper/2303.01193/) (2023)
- [RemoCap: Disentangled Representation Learning for Motion Capture](https://mlsanity.com/paper/2405.12724/) (2024)
- [Graph Edge Representation via Tensor Product Graph Convolutional Representation](https://mlsanity.com/paper/2406.14846/) (2024)
- [BetaExplainer: A Probabilistic Method to Explain Graph Neural Networks](https://mlsanity.com/paper/2412.11964/) (2024)
- [Weighted Graph Structure Learning with Attention Denoising for Node Classification](https://mlsanity.com/paper/2503.12157/) (2025)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have proved to be an effective representation learning framework for graph-structured data, and have achieved state-of-the-art performance on many practical predictive tasks, such as node classification, link prediction and graph classification. Among the variants of GNNs, Graph Attention Networks (GATs) learn to assign dense attention coefficients over all neighbors of a node for feature aggregation, and improve the performance of many graph learning tasks. However, real-world graphs are often very large and noisy, and GATs are prone to overfitting if not regularized properly. Even worse, the local aggregation mechanism of GATs may fail on disassortative graphs, where nodes within local neighborhood provide more noise than useful information for feature aggregation. In this paper, we propose Sparse Graph Attention Networks (SGATs) that learn sparse attention coefficients under an L0-norm regularization, and the learned sparse attentions are then used for all GNN layers, resulting in an edge-sparsified graph. By doing so, we can identify noisy/task-irrelevant edges, and thus perform feature aggregation on most informative neighbors. Extensive experiments on synthetic and real-world graph learning benchmarks demonstrate the superior performance of SGATs. In particular, SGATs can remove about 50%-80% edges from large assortative graphs, while retaining similar classification accuracies. On disassortative graphs, SGATs prune majority of noisy edges and outperform GATs in classification accuracies by significant margins. Furthermore, the removed edges can be interpreted intuitively and quantitatively. To the best of our knowledge, this is the first graph learning algorithm that shows significant redundancies in graphs and edge-sparsified graphs can achieve similar or sometimes higher predictive performances than original graphs.