# GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding

arXiv: https://arxiv.org/abs/1903.00757 · HTML: https://mlsanity.com/paper/1903.00757/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2019.

## Built on by (most cited first, 30 of 40)

- [Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Prediction](https://mlsanity.com/paper/2106.06935/) (2021)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Sequence-to-Sequence Knowledge Graph Completion and Question Answering](https://mlsanity.com/paper/2203.10321/) (2022)
- [Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future](https://mlsanity.com/paper/2105.13137/) (2021)
- [A*Net: A Scalable Path-based Reasoning Approach for Knowledge Graphs](https://mlsanity.com/paper/2206.04798/) (2022)
- [Marius: Learning Massive Graph Embeddings on a Single Machine](https://mlsanity.com/paper/2101.08358/) (2021)
- [CitationIE: Leveraging the Citation Graph for Scientific Information Extraction](https://mlsanity.com/paper/2106.01560/) (2021)
- [Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding](https://mlsanity.com/paper/2110.07582/) (2021)
- [SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs](https://mlsanity.com/paper/2110.14890/) (2021)
- [Knowledge Graph informed Fake News Classification via Heterogeneous Representation Ensembles](https://mlsanity.com/paper/2110.10457/) (2021)
- [Friendly Neighbors: Contextualized Sequence-to-Sequence Link Prediction](https://mlsanity.com/paper/2305.13059/) (2023)
- [Cleora: A Simple, Strong and Scalable Graph Embedding Scheme](https://mlsanity.com/paper/2102.02302/) (2021)
- [Towards Automatic Bias Detection in Knowledge Graphs](https://mlsanity.com/paper/2109.10697/) (2021)
- [A Retrieve-and-Read Framework for Knowledge Graph Link Prediction](https://mlsanity.com/paper/2212.09724/) (2022)
- [FairMILE: Towards an Efficient Framework for Fair Graph Representation Learning](https://mlsanity.com/paper/2211.09925/) (2022)
- [Aligning Knowledge Graphs and Language Models for Factual Accuracy](https://mlsanity.com/paper/2507.13411/) (2025)
- [HDReason: Algorithm-Hardware Codesign for Hyperdimensional Knowledge Graph Reasoning](https://mlsanity.com/paper/2403.05763/) (2024)
- [AutoML-guided Fusion of Entity and LLM-based Representations for Document Classification](https://mlsanity.com/paper/2408.09794/) (2024)
- [TwERC: High Performance Ensembled Candidate Generation for Ads Recommendation at Twitter](https://mlsanity.com/paper/2302.13915/) (2023)
- [Distributed Graph Embedding with Information-Oriented Random Walks](https://mlsanity.com/paper/2303.15702/) (2023)
- [GraphextQA: A Benchmark for Evaluating Graph-Enhanced Large Language Models](https://mlsanity.com/paper/2310.08487/) (2023)
- [HeteroMILE: a Multi-Level Graph Representation Learning Framework for Heterogeneous Graphs](https://mlsanity.com/paper/2404.00816/) (2024)
- [Graph4Rec: A Universal Toolkit with Graph Neural Networks for Recommender Systems](https://mlsanity.com/paper/2112.01035/) (2021)
- [Bi-Link: Bridging Inductive Link Predictions from Text via Contrastive Learning of Transformers and Prompts](https://mlsanity.com/paper/2210.14463/) (2022)
- [NeuralKG-ind: A Python Library for Inductive Knowledge Graph Representation Learning](https://mlsanity.com/paper/2304.14678/) (2023)
- [HUGE: Huge Unsupervised Graph Embeddings with TPUs](https://mlsanity.com/paper/2307.14490/) (2023)
- [Large language models converge toward human-like concept organization](https://mlsanity.com/paper/2308.15047/) (2023)
- [Enhancing PyKEEN with Multiple Negative Sampling Solutions for Knowledge Graph Embedding Models](https://mlsanity.com/paper/2508.05587/) (2025)
- [Multi-Faceted Large Embedding Tables for Pinterest Ads Ranking](https://mlsanity.com/paper/2508.05700/) (2025)
- [Scalable Graph Embedding LearningOn A Single GPU](https://mlsanity.com/paper/2110.06991/) (2021)

## Abstract (arXiv)

Learning continuous representations of nodes is attracting growing interest in both academia and industry recently, due to their simplicity and effectiveness in a variety of applications. Most of existing node embedding algorithms and systems are capable of processing networks with hundreds of thousands or a few millions of nodes. However, how to scale them to networks that have tens of millions or even hundreds of millions of nodes remains a challenging problem. In this paper, we propose GraphVite, a high-performance CPU-GPU hybrid system for training node embeddings, by co-optimizing the algorithm and the system. On the CPU end, augmented edge samples are parallelly generated by random walks in an online fashion on the network, and serve as the training data. On the GPU end, a novel parallel negative sampling is proposed to leverage multiple GPUs to train node embeddings simultaneously, without much data transfer and synchronization. Moreover, an efficient collaboration strategy is proposed to further reduce the synchronization cost between CPUs and GPUs. Experiments on multiple real-world networks show that GraphVite is super efficient. It takes only about one minute for a network with 1 million nodes and 5 million edges on a single machine with 4 GPUs, and takes around 20 hours for a network with 66 million nodes and 1.8 billion edges. Compared to the current fastest system, GraphVite is about 50 times faster without any sacrifice on performance.