mlsanityData to 14 Oct 2026

GNNear: Accelerating Full-Batch Training of Graph Neural Networks with Near-Memory Processing

arXiv 2111.00680 ↗First posted 1 Nov 2021cs.LG cs.ARcited by 6

Not on the map. Why?

Abstract (arXiv)

Recently, Graph Neural Networks (GNNs) have become state-of-the-art algorithms for analyzing non-euclidean graph data. However, to realize efficient GNN training is challenging, especially on large graphs. The reasons are many-folded: 1) GNN training incurs a substantial memory footprint. Full-batch training on large graphs even requires hundreds to thousands of gigabytes of memory. 2) GNN training involves both memory-intensive and computation-intensive operations, challenging current CPU/GPU platforms. 3) The irregularity of graphs can result in severe resource under-utilization and load-imbalance problems. This paper presents a GNNear accelerator to tackle these challenges. GNNear adopts a DIMM-based memory system to provide sufficient memory capacity. To match the heterogeneous nature of GNN training, we offload the memory-intensive Reduce operations to in-DIMM Near-Memory-Engines (NMEs), making full use of the high aggregated local bandwidth. We adopt a Centralized-Acceleration-Engine (CAE) to process the computation-intensive Update operations. We further propose several optimization strategies to deal with the irregularity of input graphs and improve GNNear's performance. Comprehensive evaluations on 16 GNN training tasks demonstrate that GNNear achieves 30.8×/2.5× geomean speedup and 79.6×/7.3×(geomean) higher energy efficiency compared to Xeon E5-2698-v4 CPU and NVIDIA V100 GPU.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 52 of its 121 references that are in the map. First posted on arXiv 1 Nov 2021.

Builds on

52 of 121 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2016Densely Connected Convolutional Networkscited by 6,720
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2018How Powerful are Graph Neural Networks?cited by 4,393
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  11. 2015Fast R-CNNcited by 2,933
  12. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
38 more
  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  2. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,525
  3. 2016YOLO9000: Better, Faster, Strongercited by 1,431
  4. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  5. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  6. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  7. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  8. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  9. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  10. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  11. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  12. 2018T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Predictioncited by 401
  13. 2019KGAT: Knowledge Graph Attention Network for Recommendationcited by 343
  14. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 335
  15. 2018Session-based Recommendation with Graph Neural Networkscited by 312
  16. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  17. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  18. 2018Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecastingcited by 146
  19. 2019Grid-GCN for Fast and Scalable Point Cloud Learningcited by 103
  20. 2019PU-GCN: Point Cloud Upsampling using Graph Convolutional Networkscited by 101
  21. 2020GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learningcited by 89
  22. 2019The Architectural Implications of Facebook's DNN-based Personalized Recommendationcited by 83
  23. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  24. 2020HyGCN: A GCN Accelerator with Hybrid Architecturecited by 74
  25. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  26. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  27. 2020Reducing Communication in Graph Neural Network Trainingcited by 50
  28. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  29. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 43
  30. 2019EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networkscited by 43
  31. 2021Sampling methods for efficient training of graph convolutional networks: A surveycited by 40
  32. 2017Learning Graph Representations with Embedding Propagationcited by 38
  33. 2019GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platformscited by 31
  34. 2021Marius: Learning Massive Graph Embeddings on a Single Machinecited by 29
  35. 2019IntentGC: a Scalable Graph Convolution Framework Fusing Heterogeneous Information for Recommendationcited by 19
  36. 2021BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matricescited by 11
  37. 2020Rubik: A Hierarchical Architecture for Efficient Graph Learningcited by 7
  38. 2018A Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasetscited by 5

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 2.

6 papers.

  1. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  2. 2024PyGim: An Efficient Graph Neural Network Library for Real Processing-In-Memory Architecturescited by 3
  3. 2023GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusterscited by 2
  4. 2025DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architecturescited by 0
  5. 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inferencecited by 0
  6. 2026LGNNIC: Acceleration of Large-Scale GNN Training using SmartNICscited by 0