mlsanityData to 7 Oct 2026

PyGim: An Efficient Graph Neural Network Library for Real Processing-In-Memory Architectures

arXiv 2402.16731 ↗First posted 26 Feb 2024cs.AR cs.DC cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) are emerging ML models to analyze graph-structure data. Graph Neural Network (GNN) execution involves both compute-intensive and memory-intensive kernels, the latter dominates the total time, being significantly bottlenecked by data movement between memory and processors. Processing-In-Memory (PIM) systems can alleviate this data movement bottleneck by placing simple processors near or inside to memory arrays. In this work, we introduce PyGim, an efficient ML library that accelerates GNNs on real PIM systems. We propose intelligent parallelization techniques for memory-intensive kernels of GNNs tailored for real PIM systems, and develop handy Python API for them. We provide hybrid GNN execution, in which the compute-intensive and memory-intensive kernels are executed in processor-centric and memory-centric computing systems, respectively. We extensively evaluate PyGim on a real-world PIM system with 1992 PIM cores using emerging GNN models, and demonstrate that it outperforms its state-of-the-art CPU counterpart on Intel Xeon by on average 3.04x, and achieves higher resource utilization than CPU and GPU systems. Our work provides useful recommendations for software, system and hardware designers. PyGim is publicly available at https://github.com/CMU-SAFARI/PyGim.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 56 of its 144 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

56 of 144 references.

References · 56

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  7. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  8. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  9. 2019Graph Neural Networks for Social Recommendationcited by 711
  10. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  11. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  12. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
38 more
  1. 2018Session-based Recommendation with Graph Neural Networkscited by 312
  2. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 287
  3. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  4. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
  5. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  6. 2018Learning to Optimize Tensor Programscited by 138
  7. 2020Sparse GPU Kernels for Deep Learningcited by 138
  8. 2019PU-GCN: Point Cloud Upsampling using Graph Convolutional Networkscited by 100
  9. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  10. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  11. 2020Reducing Communication in Graph Neural Network Trainingcited by 50
  12. 2021BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessingcited by 50
  13. 2019RecNMP: Accelerating Personalized Recommendation with Near-Memory Processingcited by 49
  14. 2022SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learningcited by 47
  15. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 42
  16. 2019EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networkscited by 42
  17. 2020GRIP: A Graph Neural Network Accelerator Architecturecited by 37
  18. 2021A Deep Learning Based Cost Model for Automatic Code Optimizationcited by 33
  19. 2020A Modern Primer on Processing in Memorycited by 30
  20. 2021A Runtime-Based Computational Performance Predictor for Deep Neural Network Trainingcited by 29
  21. 2019Processing Data Where It Makes Sense: Enabling In-Memory Computationcited by 21
  22. 2019QPyTorch: A Low-Precision Arithmetic Simulation Frameworkcited by 21
  23. 2022Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Cachingcited by 21
  24. 2018Design Principles for Sparse Matrix Multiplication on the GPUcited by 18
  25. 2022TLP: A Deep Learning-based Cost Model for Tensor Program Tuningcited by 14
  26. 2023Pathfinding Future PIM Architectures by Demystifying a Commercial PIM Technologycited by 13
  27. 2022SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architecturescited by 12
  28. 2020NATSA: A Near-Data Processing Accelerator for Time Series Analysiscited by 10
  29. 2021SynCron: Efficient Synchronization Support for Near-Data-Processing Architecturescited by 9
  30. 2021Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecturecited by 9
  31. 2017TensorQuant - A Simulation Toolbox for Deep Neural Network Quantizationcited by 8
  32. 2019SMASH: Co-designing Software Compression and Hardware-Accelerated Indexing for Efficient Sparse Matrix Operationscited by 7
  33. 2021Sextans: A Streaming Accelerator for General-Purpose Sparse-Matrix Dense-Matrix Multiplicationcited by 6
  34. 2021GNNear: Accelerating Full-Batch Training of Graph Neural Networks with Near-Memory Processingcited by 6
  35. 2022A Framework for High-throughput Sequence Alignment using Real Processing-in-Memory Systemscited by 5
  36. 2021GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networkscited by 4
  37. 2023SimplePIM: A Software Framework for Productive and Efficient Processing-in-Memorycited by 4
  38. 2020Bandwidth-Optimized Parallel Algorithms for Sparse Matrix-Matrix Multiplication using Propagation Blockingcited by 3

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory Systemcited by 5
  2. 2024SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systemscited by 1