mlsanityData to 7 Oct 2026

Tango: rethinking quantization for graph neural network training on GPUs

arXiv 2308.00890 ↗First posted 2 Aug 2023cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) are becoming increasingly popular due to their superior performance in critical graph-related tasks. While quantization is widely used to accelerate GNN computation, quantized training faces unprecedented challenges. Current quantized GNN training systems often have longer training times than their full-precision counterparts for two reasons: (i) addressing the accuracy challenge leads to excessive overhead, and (ii) the optimization potential exposed by quantization is not adequately leveraged. This paper introduces Tango which re-thinks quantization challenges and opportunities for graph neural network training on GPUs with three contributions: Firstly, we introduce efficient rules to maintain accuracy during quantized GNN training. Secondly, we design and implement quantization-aware primitives and inter-primitive optimizations that can speed up GNN training. Finally, we integrate Tango with the popular Deep Graph Library (DGL) system and demonstrate its superior performance over state-of-the-art approaches on various GNN models and datasets.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 55 of its 106 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

55 of 106 references.

References · 55

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  8. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  9. 2020Heterogeneous Graph Transformercited by 581
  10. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  11. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  12. 2021A White Paper on Neural Network Quantizationcited by 453
38 more
  1. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  2. 2015Deep Learning with Limited Numerical Precisioncited by 368
  3. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  4. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  5. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  6. 2020The Surprising Power of Graph Neural Networks with Random Node Initializationcited by 190
  7. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
  8. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  9. 2018Learning to Optimize Tensor Programscited by 138
  10. 2015A Survey of Heterogeneous Information Network Analysiscited by 109
  11. 2020Degree-Quant: Quantization-Aware Training for Graph Neural Networkscited by 93
  12. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  13. 2019Towards Unified INT8 Training for Convolutional Neural Networkcited by 77
  14. 2020DGL-KE: Training Knowledge Graph Embeddings at Scalecited by 67
  15. 2017Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detectioncited by 65
  16. 2019AdaBits: Neural Network Quantization with Adaptive Bit-Widthscited by 63
  17. 2021ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Trainingcited by 61
  18. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  19. 2019Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Modelcited by 58
  20. 2018Mixed Precision Training of Convolutional Neural Networks using Integer Operationscited by 45
  21. 2017Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networkscited by 43
  22. 2020GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networkscited by 42
  23. 2019SWALP : Stochastic Weight Averaging in Low-Precision Trainingcited by 40
  24. 2021VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantizationcited by 39
  25. 2020FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systemscited by 36
  26. 2019Asynchronous Decentralized SGD with Quantized and Local Updatescited by 35
  27. 2020Binary Graph Neural Networkscited by 32
  28. 2022GACT: Activation Compressed Training for Generic Network Architecturescited by 31
  29. 2020SGQuant: Squeezing the Last Bit on Graph Neural Networks with Specialized Quantizationcited by 25
  30. 2020Efficient Algorithms for Device Placement of DNN Graph Operatorscited by 24
  31. 2021Motif Prediction with Graph Neural Networkscited by 18
  32. 2021QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Corecited by 18
  33. 2018Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithmcited by 15
  34. 2020C-SAW: A Framework for Graph Sampling and Random Walk on GPUscited by 13
  35. 2018Scrambled Linear Pseudorandom Number Generatorscited by 12
  36. 2022Motif-based Graph Representation Learning with Application to Chemical Moleculescited by 9
  37. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7
  38. 2022TinyKG: Memory-Efficient Training Framework for Knowledge Graph Neural Recommender Systemscited by 7

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 2.

4 papers.

  1. 2023Graph Meets LLMs: Towards Large Graph Modelscited by 19
  2. 2025Deal: Distributed End-to-End GNN Inference for All Nodescited by 1
  3. 2026TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Trainingcited by 1
  4. 2026LGNNIC: Acceleration of Large-Scale GNN Training using SmartNICscited by 0