# Cleora: A Simple, Strong and Scalable Graph Embedding Scheme

arXiv: https://arxiv.org/abs/2102.02302 · HTML: https://mlsanity.com/paper/2102.02302/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 50 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 19 of 19)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [LINE: Large-scale Information Network Embedding](https://mlsanity.com/paper/1503.03578/) (2015)
- [Multi-scale Attributed Node Embedding](https://mlsanity.com/paper/1909.13021/) (2019)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [PyTorch-BigGraph: A Large-scale Graph Embedding System](https://mlsanity.com/paper/1903.12287/) (2019)
- [Inferring Networks of Substitutable and Complementary Products](https://mlsanity.com/paper/1506.08839/) (2015)
- [HARP: Hierarchical Representation Learning for Networks](https://mlsanity.com/paper/1706.07845/) (2017)
- [VERSE: Versatile Graph Embeddings from Similarity Measures](https://mlsanity.com/paper/1803.04742/) (2018)
- [Graph Embedding on Biomedical Networks: Methods, Applications, and Evaluations](https://mlsanity.com/paper/1906.05017/) (2019)
- [GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding](https://mlsanity.com/paper/1903.00757/) (2019)
- [The Effects of Randomness on the Stability of Node Embeddings](https://mlsanity.com/paper/2005.10039/) (2020)
- [Predicting Twitter User Socioeconomic Attributes with Network and Language Information](https://mlsanity.com/paper/1804.04095/) (2018)
- [GOSH: Embedding Big Graphs on Small Hardware](https://mlsanity.com/paper/2008.12336/) (2020)

## Built on by (most cited first, 8 of 8)

- [Does a Technique for Building Multimodal Representation Matter? -- Comparative Analysis](https://mlsanity.com/paper/2206.06367/) (2022)
- [Understanding and Improvement of Adversarial Training for Network Embedding from an Optimization Perspective](https://mlsanity.com/paper/2105.08007/) (2021)
- [Temporal graph models fail to capture global temporal dynamics](https://mlsanity.com/paper/2309.15730/) (2023)
- [Modeling Multi-Destination Trips with Sketch-Based Model](https://mlsanity.com/paper/2102.11252/) (2021)
- [Identifying Substitute and Complementary Products for Assortment Optimization with Cleora Embeddings](https://mlsanity.com/paper/2208.06262/) (2022)
- [Consumer Transactions Simulation through Generative Adversarial Networks](https://mlsanity.com/paper/2408.03655/) (2024)
- [Scalable Graph Embedding LearningOn A Single GPU](https://mlsanity.com/paper/2110.06991/) (2021)
- [Graph Decipher: A transparent dual-attention graph neural network to understand the message-passing mechanism for the node classification](https://mlsanity.com/paper/2201.01381/) (2022)

## Abstract (arXiv)

The area of graph embeddings is currently dominated by contrastive learning methods, which demand formulation of an explicit objective function and sampling of positive and negative examples. This creates a conceptual and computational overhead. Simple, classic unsupervised approaches like Multidimensional Scaling (MSD) or the Laplacian eigenmap skip the necessity of tedious objective optimization, directly exploiting data geometry. Unfortunately, their reliance on very costly operations such as matrix eigendecomposition make them unable to scale to large graphs that are common in today's digital world. In this paper we present Cleora: an algorithm which gets the best of two worlds, being both unsupervised and highly scalable. We show that high quality embeddings can be produced without the popular step-wise learning framework with example sampling. An intuitive learning objective of our algorithm is that a node should be similar to its neighbors, without explicitly pushing disconnected nodes apart. The objective is achieved by iterative weighted averaging of node neigbors' embeddings, followed by normalization across dimensions. Thanks to the averaging operation the algorithm makes rapid strides across the embedding space and usually reaches optimal embeddings in just a few iterations. Cleora runs faster than other state-of-the-art CPU algorithms and produces embeddings of competitive quality as measured on downstream tasks: link prediction and node classification. We show that Cleora learns a data abstraction that is similar to contrastive methods, yet at much lower computational cost. We open-source Cleora under the MIT license allowing commercial use under https://github.com/Synerise/cleora.