# word2vec, node2vec, graph2vec, X2vec: Towards a Theory of Vector Embeddings of Structured Data

arXiv: https://arxiv.org/abs/2003.12590 · HTML: https://mlsanity.com/paper/2003.12590/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2020.

## Built on by (most cited first, 30 of 38)

- [Combinatorial optimization and reasoning with graph neural networks](https://mlsanity.com/paper/2102.09544/) (2021)
- [Reconstruction for Powerful Graph Representations](https://mlsanity.com/paper/2110.00577/) (2021)
- [Graph Neural Networks with Local Graph Parameters](https://mlsanity.com/paper/2106.06707/) (2021)
- [SpeqNets: Sparsity-aware Permutation-equivariant Graph Networks](https://mlsanity.com/paper/2203.13913/) (2022)
- [WL meet VC](https://mlsanity.com/paper/2301.11039/) (2023)
- [Fine-grained Expressivity of Graph Neural Networks](https://mlsanity.com/paper/2306.03698/) (2023)
- [Expectation-Complete Graph Representations with Homomorphisms](https://mlsanity.com/paper/2306.05838/) (2023)
- [Tensor Graph Convolutional Network for Dynamic Graph Representation Learning](https://mlsanity.com/paper/2401.07065/) (2024)
- [FSW-GNN: A Bi-Lipschitz WL-Equivalent Graph Neural Network](https://mlsanity.com/paper/2410.09118/) (2024)
- [Self-Discriminative Modeling for Anomalous Graph Detection](https://mlsanity.com/paper/2310.06261/) (2023)
- [On the H"older Stability of Multiset and Graph Neural Networks](https://mlsanity.com/paper/2406.06984/) (2024)
- [Topological Node2vec: Enhanced Graph Embedding via Persistent Homology](https://mlsanity.com/paper/2309.08241/) (2023)
- [Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision](https://mlsanity.com/paper/2402.03480/) (2024)
- [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024)
- [Molecular Machine Learning in Chemical Process Design](https://mlsanity.com/paper/2508.20527/) (2025)
- [Structural Node Embeddings with Homomorphism Counts](https://mlsanity.com/paper/2308.15283/) (2023)
- [Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applications](https://mlsanity.com/paper/2406.02601/) (2024)
- [On the Surprising Behaviour of node2vec](https://mlsanity.com/paper/2206.08252/) (2022)
- [Permutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning](https://mlsanity.com/paper/2506.20324/) (2025)
- [Accurately Modeling Biased Random Walks on Weighted Graphs Using textitNode2vec+](https://mlsanity.com/paper/2109.08031/) (2021)
- [Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning](https://mlsanity.com/paper/2305.09756/) (2023)
- [Grables: When Graph Structure Helps Tabular Prediction](https://mlsanity.com/paper/2602.03945/) (2026)
- [Network representation learning systematic review: ancestors and current development state](https://mlsanity.com/paper/2109.07583/) (2021)
- [From structure mining to unsupervised exploration of atomic octahedral networks](https://mlsanity.com/paper/2306.12272/) (2023)
- [Node Embeddings via Neighbor Embeddings](https://mlsanity.com/paper/2503.23822/) (2025)
- [Multimodal Generative AI with Autoregressive LLMs for Human Motion Understanding and Generation: A Way Forward](https://mlsanity.com/paper/2506.03191/) (2025)
- [Scaling up graph homomorphism for classification via sampling](https://mlsanity.com/paper/2104.04040/) (2021)
- [MultiSAGE: a multiplex embedding algorithm for inter-layer link prediction](https://mlsanity.com/paper/2206.13223/) (2022)
- [Estimation of embedding vectors in high dimensions](https://mlsanity.com/paper/2312.07802/) (2023)
- [Sequence graphs realizations and ambiguity in language models](https://mlsanity.com/paper/2402.08830/) (2024)

## Abstract (arXiv)

Vector representations of graphs and relational structures, whether hand-crafted feature vectors or learned representations, enable us to apply standard data analysis and machine learning techniques to the structures. A wide range of methods for generating such embeddings have been studied in the machine learning and knowledge representation literature. However, vector embeddings have received relatively little attention from a theoretical point of view. Starting with a survey of embedding techniques that have been used in practice, in this paper we propose two theoretical approaches that we see as central for understanding the foundations of vector embeddings. We draw connections between the various approaches and suggest directions for future research.