# Learning Graph Node Embeddings by Smooth Pair Sampling

arXiv: https://arxiv.org/abs/2501.12884 · HTML: https://mlsanity.com/paper/2501.12884/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 19 of 19)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec](https://mlsanity.com/paper/1710.02971/) (2017)
- [Understanding Negative Sampling in Graph Representation Learning](https://mlsanity.com/paper/2005.09863/) (2020)
- [VERSE: Versatile Graph Embeddings from Similarity Measures](https://mlsanity.com/paper/1803.04742/) (2018)
- [A Comparative Study for Unsupervised Network Representation Learning](https://mlsanity.com/paper/1903.07902/) (2019)
- [A Broader Picture of Random-walk Based Graph Embedding](https://mlsanity.com/paper/2110.12344/) (2021)
- [Fast Sequence-Based Embedding with Diffusion Graphs](https://mlsanity.com/paper/2001.07463/) (2020)
- [Attributed Network Embedding via Subspace Discovery](https://mlsanity.com/paper/1901.04095/) (2019)
- [Residual2Vec: Debiasing graph embedding with random graphs](https://mlsanity.com/paper/2110.07654/) (2021)
- [Learning Edge Representations via Low-Rank Asymmetric Projections](https://mlsanity.com/paper/1705.05615/) (2017)
- [LoNe Sampler: Graph node embeddings by coordinated local neighborhood sampling](https://mlsanity.com/paper/2211.15114/) (2022)

## Abstract (arXiv)

Random walk-based node embedding algorithms have attracted a lot of attention due to their scalability and ease of implementation. Previous research has focused on different walk strategies, optimization objectives, and embedding learning models. Inspired by observations on real data, we take a different approach and propose a new regularization technique. More precisely, the frequencies of node pairs generated by the skip-gram model on random walk node sequences follow a highly skewed distribution which causes learning to be dominated by a fraction of the pairs. We address the issue by designing an efficient sampling procedure that generates node pairs according to their em smoothed frequency. Theoretical and experimental results demonstrate the advantages of our approach.