# Fast Training of Sparse Graph Neural Networks on Dense Hardware

arXiv: https://arxiv.org/abs/1906.11786 · HTML: https://mlsanity.com/paper/1906.11786/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2019.

## Built on by (most cited first, 2 of 2)

- [Improving Graph Generation by Restricting Graph Bandwidth](https://mlsanity.com/paper/2301.10857/) (2023)
- [AutoGMap: Learning to Map Large-scale Sparse Graphs on Memristive Crossbars](https://mlsanity.com/paper/2111.07684/) (2021)

## Abstract (arXiv)

Graph neural networks have become increasingly popular in recent years due to their ability to naturally encode relational input data and their ability to scale to large graphs by operating on a sparse representation of graph adjacency matrices. As we look to scale up these models using custom hardware, a natural assumption would be that we need hardware tailored to sparse operations and/or dynamic control flow. In this work, we question this assumption by scaling up sparse graph neural networks using a platform targeted at dense computation on fixed-size data. Drawing inspiration from optimization of numerical algorithms on sparse matrices, we develop techniques that enable training the sparse graph neural network model from Allamanis et al. [2018] in 13 minutes using a 512-core TPUv2 Pod, whereas the original training takes almost a day.