mlsanityData to 7 Oct 2026

TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs

arXiv 2112.02052 ↗First posted 3 Dec 2021cs.LG cs.DCcited by 17

Not on the map. Why?

Abstract (arXiv)

Recently, graph neural networks (GNNs), as the backbone of graph-based machine learning, demonstrate great success in various domains (e.g., e-commerce). However, the performance of GNNs is usually unsatisfactory due to the highly sparse and irregular graph-based operations. To this end, we propose TC-GNN, the first GNN acceleration framework based on GPU Tensor Core Units (TCUs). The core idea is to reconcile the "Sparse" GNN computation with the high-performance "Dense" TCUs. Specifically, we conduct an in-depth analysis of the sparse operations in mainstream GNN computing frameworks. We introduce a novel sparse graph translation technique to facilitate TCU processing of the sparse GNN workload. We implement an effective CUDA core and TCU collaboration design to fully utilize GPU resources. We integrate TC-GNN with the PyTorch framework for high programmability. Rigorous experiments show an average of 1.70X speedup over the state-of-the-art DGL framework across various models and datasets.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 23 of its 70 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

23 of 70 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  9. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  10. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  11. 2018Link Prediction Based on Graph Neural Networkscited by 829
  12. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
11 more
  1. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  2. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  3. 2018Attention-based Graph Neural Network for Semi-supervised Learningcited by 100
  4. 2020GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networkscited by 42
  5. 2017Learning Graph Representations with Embedding Propagationcited by 38
  6. 2021QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Corecited by 18
  7. 2021A Broader Picture of Random-walk Based Graph Embeddingcited by 15
  8. 2020Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUscited by 12
  9. 2021PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accessescited by 12
  10. 2017Deep Feature Learning via Structured Graph Laplacian Embedding for Person Re-Identificationcited by 9
  11. 2020Accelerating Sparse Matrix-Matrix Multiplication with GPU Tensor Corescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 6; 2026: 3.

17 papers.

Sort:

By area:

  1. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  2. 2025DSV: Exploiting Dynamic Sparsity to Accelerate Large-Scale Video DiT Trainingcited by 15
  3. 2023GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelismcited by 6
  4. 2024DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Trainingcited by 5
  5. 2024GNNBENCH: Fair and Productive Benchmarking for Single-GPU GNN Systemcited by 3
  6. 2024Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Corescited by 3
  7. 2024FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Corescited by 2
  8. 2025Fused3S: Fast Sparse Attention on Tensor Corescited by 2
  9. 2024Single-GPU GNN Systems: Traps and Pitfallscited by 1
  10. 2024Optimization of GNN Training Through Half-precisioncited by 1
  11. 2024Accelerating Sparse Graph Neural Networks with Tensor Core Optimizationcited by 1
  12. 2025Libra: Unleashing GPU Heterogeneity for High-Performance Sparse Matrix Multiplicationcited by 1
5 more
  1. 2025APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Accelerationcited by 1
  2. 2025AutoSAGE: Input-Aware CUDA Scheduling for Sparse GNN Aggregation (SpMM/SDDMM) and CSR Attentioncited by 0
  3. 2026On Efficient Scaling of GNNs via IO-Aware Layers Implementationscited by 0
  4. 2026Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matricescited by 0
  5. 2026SparseDitto: An Agentic Sparse Compilation Framework through Architecture-Aware Synthesis on GPUscited by 0