# Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification

arXiv: https://arxiv.org/abs/2006.08796 · HTML: https://mlsanity.com/paper/2006.08796/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 16 of 16)

- [NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification](https://mlsanity.com/paper/2306.08385/) (2023)
- [Survey on Graph Neural Network Acceleration: An Algorithmic Perspective](https://mlsanity.com/paper/2202.04822/) (2022)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Sketch-GNN: Scalable Graph Neural Networks with Sublinear Training Complexity](https://mlsanity.com/paper/2406.15575/) (2024)
- [Sparse Covariance Neural Networks](https://mlsanity.com/paper/2410.01669/) (2024)
- [Efficient Topology-aware Data Augmentation for High-Degree Graph Neural Networks](https://mlsanity.com/paper/2406.05482/) (2024)
- [Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study](https://mlsanity.com/paper/2409.11129/) (2024)
- [Unifews: You Need Fewer Operations for Efficient Graph Neural Networks](https://mlsanity.com/paper/2403.13268/) (2024)
- [LSP : Acceleration and Regularization of Graph Neural Networks via Locality Sensitive Pruning of Graphs](https://mlsanity.com/paper/2111.05694/) (2021)
- [Rethinking Efficiency and Redundancy in Training Large-scale Graphs](https://mlsanity.com/paper/2209.00800/) (2022)
- [On the Feasibility of Fidelity^- for Graph Pruning](https://mlsanity.com/paper/2406.11504/) (2024)
- [UEChecker: Detecting Unchecked External Call Vulnerabilities in DApps via Graph Analysis](https://mlsanity.com/paper/2508.01343/) (2025)
- [Graph Enabled Cross-Domain Knowledge Transfer](https://mlsanity.com/paper/2304.03452/) (2023)
- [Fast and Effective GNN Training through Sequences of Random Path Graphs](https://mlsanity.com/paper/2306.04828/) (2023)
- [Infinite Width Graph Neural Networks for Node Regression/ Classification](https://mlsanity.com/paper/2310.08176/) (2023)
- [Theoretical Learning Performance of Graph Neural Networks: The Impact of Jumping Connections and Layer-wise Sparsification](https://mlsanity.com/paper/2507.05533/) (2025)

## Abstract (arXiv)

The attention mechanism has demonstrated superior performance for inference over nodes in graph neural networks (GNNs), however, they result in a high computational burden during both training and inference. We propose FastGAT, a method to make attention based GNNs lightweight by using spectral sparsification to generate an optimal pruning of the input graph. This results in a per-epoch time that is almost linear in the number of graph nodes as opposed to quadratic. We theoretically prove that spectral sparsification preserves the features computed by the GAT model, thereby justifying our algorithm. We experimentally evaluate FastGAT on several large real world graph datasets for node classification tasks under both inductive and transductive settings. FastGAT can dramatically reduce (up to 10x) the computational time and memory requirements, allowing the usage of attention based GNNs on large graphs.