# Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graph

arXiv: https://arxiv.org/abs/2208.05648 · HTML: https://mlsanity.com/paper/2208.05648/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 11 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [Positional Encoding to Control Output Sequence Length](https://mlsanity.com/paper/1904.07418/) (2019)
- [Hash Embeddings for Efficient Word Representations](https://mlsanity.com/paper/1709.03933/) (2017)
- [Differentiable Product Quantization for End-to-End Embedding Compression](https://mlsanity.com/paper/1908.09756/) (2019)
- [Compressing Word Embeddings via Deep Compositional Code Learning](https://mlsanity.com/paper/1711.01068/) (2017)
- [Learning to Hash with Graph Neural Networks for Recommender Systems](https://mlsanity.com/paper/2003.01917/) (2020)
- [On Node Features for Graph Neural Networks](https://mlsanity.com/paper/1911.08795/) (2019)
- [All Word Embeddings from One Embedding](https://mlsanity.com/paper/2004.12073/) (2020)
- [Merchant Category Identification Using Credit Card Transactions](https://mlsanity.com/paper/2011.02602/) (2020)
- [Towards a Flexible Embedding Learning Framework](https://mlsanity.com/paper/2009.10989/) (2020)

## Built on by (most cited first, 10 of 10)

- [Toward a Foundation Model for Time Series Data](https://mlsanity.com/paper/2310.03916/) (2023)
- [An Efficient Content-based Time Series Retrieval System](https://mlsanity.com/paper/2310.03919/) (2023)
- [Graph Learning and Its Advancements on Large Language Models: A Holistic Survey](https://mlsanity.com/paper/2212.08966/) (2022)
- [Sharpness-Aware Graph Collaborative Filtering](https://mlsanity.com/paper/2307.08910/) (2023)
- [Sketching Multidimensional Time Series for Fast Discord Mining](https://mlsanity.com/paper/2311.03393/) (2023)
- [Multitask Learning for Time Series Data with 2D Convolution](https://mlsanity.com/paper/2310.03925/) (2023)
- [Adversarial Collaborative Filtering for Free](https://mlsanity.com/paper/2308.13541/) (2023)
- [Temporal Treasure Hunt: Content-based Time Series Retrieval System for Discovering Insights](https://mlsanity.com/paper/2311.02560/) (2023)
- [Ego-Network Transformer for Subsequence Classification in Time Series Data](https://mlsanity.com/paper/2311.02561/) (2023)
- [Time Series Synthesis Using the Matrix Profile for Anonymization](https://mlsanity.com/paper/2311.02563/) (2023)

## Abstract (arXiv)

Graph neural networks (GNNs) are deep learning models designed specifically for graph data, and they typically rely on node features as the input to the first layer. When applying such a type of network on the graph without node features, one can extract simple graph-based node features (e.g., number of degrees) or learn the input node representations (i.e., embeddings) when training the network. While the latter approach, which trains node embeddings, more likely leads to better performance, the number of parameters associated with the embeddings grows linearly with the number of nodes. It is therefore impractical to train the input node embeddings together with GNNs within graphics processing unit (GPU) memory in an end-to-end fashion when dealing with industrial-scale graph data. Inspired by the embedding compression methods developed for natural language processing (NLP) tasks, we develop a node embedding compression method where each node is compactly represented with a bit vector instead of a floating-point vector. The parameters utilized in the compression method can be trained together with GNNs. We show that the proposed node embedding compression method achieves superior performance compared to the alternatives.