mlsanityData to 13 Oct 2026

Analysis and Optimization of GNN-Based Recommender Systems on Persistent Memory

arXiv 2207.11918 ↗First posted 25 Jul 2022cs.IR cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs), which have emerged as an effective method for handling machine learning tasks on graphs, bring a new approach to building recommender systems, where the task of recommendation can be formulated as the link prediction problem on user-item bipartite graphs. Training GNN-based recommender systems (GNNRecSys) on large graphs incurs a large memory footprint, easily exceeding the DRAM capacity on a typical server. Existing solutions resort to distributed subgraph training, which is inefficient due to the high cost of dynamically constructing subgraphs and significant redundancy across subgraphs. The emerging persistent memory technologies provide a significantly larger memory capacity than DRAMs at an affordable cost, making single-machine GNNRecSys training feasible, which eliminates the inefficiencies in distributed training. One major concern of using persistent memory devices for GNNRecSys is their relatively low bandwidth compared with DRAMs. This limitation can be particularly detrimental to achieving high performance for GNNRecSys workloads since their dominant compute kernels are sparse and memory access intensive. To understand whether persistent memory is a good fit for GNNRecSys training, we perform an in-depth characterization of GNNRecSys workloads and a comprehensive analysis of their performance on a persistent memory device, namely, Intel Optane. Based on the analysis, we provide guidance on how to configure Optane for GNNRecSys workloads. Furthermore, we present techniques for large-batch training to fully realize the advantages of single-machine GNNRecSys training. Our experiment results show that with the tuned batch size and optimal system configuration, Optane-based single-machine GNNRecSys training outperforms distributed training by a large margin, especially when handling deep GNN models.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 36 of its 66 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

36 of 66 references.

References · 36

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,318
  2. 2017Graph Attention Networkscited by 8,888
  3. 2017Inductive Representation Learning on Large Graphscited by 6,303
  4. 2017Neural Message Passing for Quantum Chemistrycited by 3,471
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,467
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,915
  7. 2014DeepWalk: Online Learning of Social Representationscited by 1,544
  8. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,488
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,353
  10. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,135
  11. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,135
  12. 2017Neural Collaborative Filteringcited by 1,110
24 more
  1. 2016Wide & Deep Learning for Recommender Systemscited by 746
  2. 2019Graph Neural Networks for Social Recommendationcited by 714
  3. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 673
  4. 2019Neural Graph Collaborative Filteringcited by 660
  5. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  6. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 518
  7. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
  8. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  9. 2018An Empirical Model of Large-Batch Trainingcited by 274
  10. 2014One weird trick for parallelizing convolutional neural networkscited by 264
  11. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  12. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
  13. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  14. 2017Fast Matrix Factorization for Online Recommendation with Implicit Feedbackcited by 101
  15. 2021Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Modelscited by 78
  16. 2020HyGCN: A GCN Accelerator with Hybrid Architecturecited by 74
  17. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  18. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  19. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  20. 2020FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systemscited by 36
  21. 2022TPP: Transparent Page Placement for CXL-Enabled Tiered-Memorycited by 10
  22. 2019Basic Performance Measurements of the Intel Optane DC Persistent Memory Modulecited by 9
  23. 2019Scalable Realistic Recommendation Datasets through Fractal Expansionscited by 7
  24. 2019An Empirical Guide to the Behavior and Use of Scalable Persistent Memorycited by 5

Built on by

Not cited yet.