# LLMPerf: GPU Performance Modeling meets Large Language Models

arXiv: https://arxiv.org/abs/2503.11244 · HTML: https://mlsanity.com/paper/2503.11244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 6 of 6)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [A Simple Model for Portable and Fast Prediction of Execution Time and Power Consumption of GPU Kernels](https://mlsanity.com/paper/2001.07104/) (2020)
- [BenchPress: A Deep Active Benchmark Generator](https://mlsanity.com/paper/2208.06555/) (2022)

## Abstract (arXiv)

Performance modeling, a pivotal domain in program cost analysis, currently relies on manually crafted models constrained by various program and hardware limitations, especially in the intricate landscape of GPGPU. Meanwhile, Large Language Models (LLMs) have demonstrated their effectiveness in addressing diverse programming challenges. Our work establishes a connection between LLMs and performance modeling, employing the LLM as a performance estimator. Through experimental exploration with carefully designed large-scale OpenCL datasets, we highlight the potential capability as well as the main difficulties of using LLMs in handling performance modeling tasks for OpenCL device source programs. As the first study for this line of work, our LLM-based performance model achieves a mean absolute percentage error of 24.25% for a large-scale generated validation set. On a set of publicly available OpenCL programs, our model achieves a mean absolute percentage error of 46.1%.