# Accurate, Efficient and Scalable Graph Embedding

arXiv: https://arxiv.org/abs/1810.11899 · HTML: https://mlsanity.com/paper/1810.11899/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2018.

## Built on by (most cited first, 30 of 35)

- [Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods](https://mlsanity.com/paper/2110.14446/) (2021)
- [Decoupling the Depth and Scope of Graph Neural Networks](https://mlsanity.com/paper/2201.07858/) (2022)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Accelerating Large Scale Real-Time GNN Inference using Channel Pruning](https://mlsanity.com/paper/2105.04528/) (2021)
- [Sampling methods for efficient training of graph convolutional networks: A survey](https://mlsanity.com/paper/2103.05872/) (2021)
- [Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets](https://mlsanity.com/paper/2103.00794/) (2021)
- [Survey on Graph Neural Network Acceleration: An Algorithmic Perspective](https://mlsanity.com/paper/2202.04822/) (2022)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Scalable algorithms for physics-informed neural and graph networks](https://mlsanity.com/paper/2205.08332/) (2022)
- [Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks](https://mlsanity.com/paper/2107.10234/) (2021)
- [Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks](https://mlsanity.com/paper/2207.08629/) (2022)
- [G-CoS: GNN-Accelerator Co-Search Towards Both Better Accuracy and Efficiency](https://mlsanity.com/paper/2109.08983/) (2021)
- [A Comprehensive Survey on Distributed Training of Graph Neural Networks](https://mlsanity.com/paper/2211.05368/) (2022)
- [GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design](https://mlsanity.com/paper/2112.11594/) (2021)
- [Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch](https://mlsanity.com/paper/2406.00552/) (2024)
- [Large-Scale Network Embedding in Apache Spark](https://mlsanity.com/paper/2106.10620/) (2021)
- [GNNSampler: Bridging the Gap between Sampling Algorithms of GNN and Hardware](https://mlsanity.com/paper/2108.11571/) (2021)
- [ASWT-SGNN: Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network](https://mlsanity.com/paper/2312.05736/) (2023)
- [DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training](https://mlsanity.com/paper/2405.05231/) (2024)
- [Binary Graph Convolutional Network with Capacity Exploration](https://mlsanity.com/paper/2210.13149/) (2022)
- [BitGNN: Unleashing the Performance Potential of Binary Graph Neural Networks on GPUs](https://mlsanity.com/paper/2305.02522/) (2023)
- [A Comprehensive Analytical Survey on Unsupervised and Semi-Supervised Graph Representation Learning Methods](https://mlsanity.com/paper/2112.10372/) (2021)
- [Local2Global: A distributed approach for scaling representation learning on graphs](https://mlsanity.com/paper/2201.04729/) (2022)
- [BatchGNN: Efficient CPU-Based Distributed GNN Training on Very Large Graphs](https://mlsanity.com/paper/2306.13814/) (2023)
- [Impression-Informed Multi-Behavior Recommender System: A Hierarchical Graph Attention Approach](https://mlsanity.com/paper/2309.03169/) (2023)
- [Local2Global: Scaling global representation learning on graphs via local training](https://mlsanity.com/paper/2107.12224/) (2021)
- [Rethinking Efficiency and Redundancy in Training Large-scale Graphs](https://mlsanity.com/paper/2209.00800/) (2022)
- [FALCON: Feature-Label Constrained Graph Net Collapse for Memory Efficient GNNs](https://mlsanity.com/paper/2312.16542/) (2023)
- [GeoAI-Enhanced Community Detection on Spatial Networks with Graph Deep Learning](https://mlsanity.com/paper/2411.15428/) (2024)
- [Optimizing Graph Transformer Networks with Graph-based Techniques](https://mlsanity.com/paper/2106.08500/) (2021)

## Abstract (arXiv)

The Graph Convolutional Network (GCN) model and its variants are powerful graph embedding tools for facilitating classification and clustering on graphs. However, a major challenge is to reduce the complexity of layered GCNs and make them parallelizable and scalable on very large graphs -- state-of the art techniques are unable to achieve scalability without losing accuracy and efficiency. In this paper, we propose novel parallelization techniques for graph sampling-based GCNs that achieve superior scalable performance on very large graphs without compromising accuracy. Specifically, our GCN guarantees work-efficient training and produces order of magnitude savings in computation and communication. To scale GCN training on tightly-coupled shared memory systems, we develop parallelization strategies for the key steps in training: For the graph sampling step, we exploit parallelism within and across multiple sampling instances, and devise an efficient data structure for concurrent accesses that provides theoretical guarantee of near-linear speedup with number of processing units. For the feature propagation step within the sampled graph, we improve cache utilization and reduce DRAM communication by data partitioning. We prove that our partitioning strategy is a 2-approximation for minimizing the communication time compared to the optimal strategy. We demonstrate that our parallel graph embedding outperforms state-of-the-art methods in scalability (with respect to number of processors, graph size and GCN model size), efficiency and accuracy on several large datasets. On a 40-core Xeon platform, our parallel training achieves 64× speedup (with AVX) in the sampling step and 25× speedup in the feature propagation step, compared to the serial implementation, resulting in a net speedup of 21×.