# PyTorch-BigGraph: A Large-scale Graph Embedding System

arXiv: https://arxiv.org/abs/1903.12287 · HTML: https://mlsanity.com/paper/1903.12287/

## Key facts

- Cited by 119 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2019.

## Built on by (most cited first, 30 of 119)

- [Graph Neural Networks for Link Prediction with Subgraph Sketching](https://mlsanity.com/paper/2209.15486/) (2022)
- [Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings](https://mlsanity.com/paper/2202.06671/) (2022)
- [NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs](https://mlsanity.com/paper/2106.12144/) (2021)
- [Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2307.04189/) (2023)
- [TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter](https://mlsanity.com/paper/2209.07562/) (2022)
- [BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling](https://mlsanity.com/paper/2203.10983/) (2022)
- [Automated Machine Learning on Graphs: A Survey](https://mlsanity.com/paper/2103.00742/) (2021)
- [Modern Baselines for SPARQL Semantic Parsing](https://mlsanity.com/paper/2204.12793/) (2022)
- [LiGNN: Graph Neural Networks at LinkedIn](https://mlsanity.com/paper/2402.11139/) (2024)
- [Marius: Learning Massive Graph Embeddings on a Single Machine](https://mlsanity.com/paper/2101.08358/) (2021)
- [CASSINI: Network-Aware Job Scheduling in Machine Learning Clusters](https://mlsanity.com/paper/2308.00852/) (2023)
- [Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge](https://mlsanity.com/paper/2101.06013/) (2021)
- [Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding](https://mlsanity.com/paper/2110.07582/) (2021)
- [MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks](https://mlsanity.com/paper/2202.02365/) (2022)
- [Saga: A Platform for Continuous Construction and Serving of Knowledge At Scale](https://mlsanity.com/paper/2204.07309/) (2022)
- [SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs](https://mlsanity.com/paper/2110.14890/) (2021)
- [GrOVe: Ownership Verification of Graph Neural Networks using Embeddings](https://mlsanity.com/paper/2304.08566/) (2023)
- [AutoGL: A Library for Automated Graph Learning](https://mlsanity.com/paper/2104.04987/) (2021)
- [Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning](https://mlsanity.com/paper/2102.04350/) (2021)
- [ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table](https://mlsanity.com/paper/2104.08542/) (2021)
- [GeoVectors: A Linked Open Corpus of OpenStreetMap Embeddings on World Scale](https://mlsanity.com/paper/2108.13092/) (2021)
- [Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directions](https://mlsanity.com/paper/2201.01288/) (2022)
- [Modeling Fine-grained Information via Knowledge-aware Hierarchical Graph for Zero-shot Entity Retrieval](https://mlsanity.com/paper/2211.10991/) (2022)
- [Friendly Neighbors: Contextualized Sequence-to-Sequence Link Prediction](https://mlsanity.com/paper/2305.13059/) (2023)
- [Negative Sampling in Knowledge Graph Representation Learning: A Review](https://mlsanity.com/paper/2402.19195/) (2024)
- [Cleora: A Simple, Strong and Scalable Graph Embedding Scheme](https://mlsanity.com/paper/2102.02302/) (2021)
- [Augmenting Knowledge Graphs for Better Link Prediction](https://mlsanity.com/paper/2203.13965/) (2022)
- [Relevant Entity Selection: Knowledge Graph Bootstrapping via Zero-Shot Analogical Pruning](https://mlsanity.com/paper/2306.16296/) (2023)
- [Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs](https://mlsanity.com/paper/2405.17846/) (2024)
- [GraphStorm: all-in-one graph machine learning framework for industry applications](https://mlsanity.com/paper/2406.06022/) (2024)

## Abstract (arXiv)

Graph embedding methods produce unsupervised node features from graphs that can then be used for a variety of machine learning tasks. Modern graphs, particularly in industrial applications, contain billions of nodes and trillions of edges, which exceeds the capability of existing embedding systems. We present PyTorch-BigGraph (PBG), an embedding system that incorporates several modifications to traditional multi-relation embedding systems that allow it to scale to graphs with billions of nodes and trillions of edges. PBG uses graph partitioning to train arbitrarily large embeddings on either a single machine or in a distributed environment. We demonstrate comparable performance with existing embedding systems on common benchmarks, while allowing for scaling to arbitrarily large graphs and parallelization on multiple machines. We train and evaluate embeddings on several large social network graphs as well as the full Freebase dataset, which contains over 100 million nodes and 2 billion edges.