mlsanityData to 14 Oct 2026

How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach?

arXiv 2609.21058 ↗First posted 17 Sep 2026cs.DC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Language models can now write GPU kernels that outperform PyTorch. We evaluate five model configurations on KernelBench level 1 and find that a frontier model produces correct kernels for 91.1% of problems and independently verified speedups on 22 of 56, including three convolutions, with a median of 1.235x. Open-weights models are far behind: the best reaches 30.4% correct with three verified speedups and solves zero convolutions. We then ask a question the literature does not: what fraction of a real model's wall clock do such kernels govern? Profiling seven workloads across three domains, we find the addressable fraction ranges from 8.9% to 58.2%. On transformers, 80-86% of runtime is spent in cuBLAS GEMM and FlashAttention, bounding realistic end-to-end improvement at roughly 1%, and the fraction shrinks with model scale. On recommenders it is 58.2%, concentrated in a single embedding kernel. We introduce DLRM-Bench, 12 recommender kernel problems in KernelBench format, and measure a 41.7% win rate at a 1.552x median there, projecting 8.63% end-to-end. Separately, we show that KernelBench's correctness check (torch.allclose with an absolute tolerance) is satisfied by a tensor of zeros on 4 of 60 level-1 problems. Two kernels in our own results exploited this before we detected them, including one scored at 283x that wrote 0.3% of its output buffer. We propose scale-invariant replacements and release all 879 evaluations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 15 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

7 of 15 references.

References · 7

  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  3. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 225
  4. 2025GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimizationcited by 22
  5. 2025ConCuR: Conciseness Makes State-of-the-Art Kernel Generationcited by 13
  6. 2026KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernelscited by 12
  7. 2026Correct but Slow: An Empirical Study of the GPU Kernel Evaluation Gap in Modern Domain-Specific Languagescited by 1

Built on by

Not cited yet.