mlsanityData to 7 Oct 2026

DIPPM: a Deep Learning Inference Performance Predictive Model using Graph Neural Networks

arXiv 2303.11733 ↗First posted 21 Mar 2023cs.PF cs.AI cs.ARcited by 1

Not on the map. Why?

Abstract (arXiv)

Deep Learning (DL) has developed to become a corner-stone in many everyday applications that we are now relying on. However, making sure that the DL model uses the underlying hardware efficiently takes a lot of effort. Knowledge about inference characteristics can help to find the right match so that enough resources are given to the model, but not too much. We have developed a DL Inference Performance Predictive Model (DIPPM) that predicts the inference latency, energy, and memory usage of a given input DL model on the NVIDIA A100 GPU. We also devised an algorithm to suggest the appropriate A100 Multi-Instance GPU profile from the output of DIPPM. We developed a methodology to convert DL models expressed in multiple frameworks to a generalized graph structure that is used in DIPPM. It means DIPPM can parse input DL models from various frameworks. Our DIPPM can be used not only helps to find suitable hardware configurations but also helps to perform rapid design-space exploration for the inference performance of a model. We constructed a graph multi-regression dataset consisting of 10,508 different DL models to train and evaluate the performance of DIPPM, and reached a resulting Mean Absolute Percentage Error (MAPE) as low as 1.9%.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

14 of 24 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  8. 2020BRP-NAS: Prediction-based NAS using GCNscited by 109
  9. 2018Predicting the Computational Cost of Deep Learning Modelscited by 49
  10. 2021A Runtime-Based Computational Performance Predictor for Deep Neural Network Trainingcited by 29
  11. 2018Relay: A New IR for Machine Learning Frameworkscited by 24
  12. 2022MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant Systems for Machine Learningcited by 15
2 more
  1. 2020Toward Accurate Platform-Aware Performance Modeling for Deep Neural Networkscited by 4
  2. 2022DNNAbacus: Toward Accurate Computational Cost Prediction for Deep Neural Networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Node-wise Feature Encoding for Neural Performance Predictioncited by 0