mlsanityData to 14 Oct 2026

Nimble GNN Embedding with Tensor-Train Decomposition

arXiv 2206.10581 ↗First posted 21 Jun 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper describes a new method for representing embedding tables of graph neural networks (GNNs) more compactly via tensor-train (TT) decomposition. We consider the scenario where (a) the graph data that lack node features, thereby requiring the learning of embeddings during training; and (b) we wish to exploit GPU platforms, where smaller tables are needed to reduce host-to-GPU communication even for large-memory GPUs. The use of TT enables a compact parameterization of the embedding, rendering it small enough to fit entirely on modern GPUs even for massive graphs. When combined with judicious schemes for initialization and hierarchical graph partitioning, this approach can reduce the size of node embedding vectors by 1,659 times to 81,362 times on large publicly available benchmark datasets, achieving comparable or better accuracy and significant speedups on multi-GPU systems. In some cases, our model without explicit node features on input can even match the accuracy of models that use node features.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 21 Jun 2022.

Builds on

20 of 30 references.

References · 20

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  5. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  6. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  7. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  8. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  9. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  10. 2020Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networkscited by 73
  11. 2019Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systemscited by 68
  12. 2020Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systemscited by 60
8 more
  1. 2021TT-Rec: Tensor Train Compression for Deep Learning Recommendation Modelscited by 58
  2. 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 56
  3. 2017Hash Embeddings for Efficient Word Representationscited by 55
  4. 2018Wide Compression: Tensor Ring Netscited by 50
  5. 2019Tensorized Embedding Layers for Efficient Model Compressioncited by 27
  6. 2017Getting deep recommenders fit: Bloom embeddings for sparse binary input/output networkscited by 14
  7. 2020Training with Multi-Layer Embeddings for Model Reductioncited by 6
  8. 2021Position-based Hash Embeddings For Scaling Graph Neural Networkscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 1.

6 papers.

  1. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 32
  2. 2024Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Modelscited by 4
  3. 2024A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systemscited by 4
  4. 2024GraphScale: A Framework to Enable Machine Learning over Billion-node Graphscited by 4
  5. 2023TensorCodec: Compact Lossy Compression of Tensors without Strong Data Assumptionscited by 0
  6. 2025Coarse-to-Fine Lightweight Meta-Embedding for ID-Based Recommendationcited by 0