# CUDA-LLM: LLMs Can Write Efficient CUDA Kernels

arXiv: https://arxiv.org/abs/2506.09092 · HTML: https://mlsanity.com/paper/2506.09092/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Builds on (most cited first, 3 of 3)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)

## Built on by (most cited first, 30 of 34)

- [CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning](https://mlsanity.com/paper/2507.14111/) (2025)
- [Astra: A Multi-Agent System for GPU Kernel Performance Optimization](https://mlsanity.com/paper/2509.07506/) (2025)
- [EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models](https://mlsanity.com/paper/2510.03760/) (2025)
- [KernelFoundry: Hardware-aware evolutionary GPU kernel optimization](https://mlsanity.com/paper/2603.12440/) (2026)
- [KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning](https://mlsanity.com/paper/2602.14293/) (2026)
- [Towards Automated Kernel Generation in the Era of LLMs](https://mlsanity.com/paper/2601.15727/) (2026)
- [AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement](https://mlsanity.com/paper/2608.20318/) (2026)
- [CUDABench: Benchmarking LLMs for Text-to-CUDA Generation](https://mlsanity.com/paper/2603.02236/) (2026)
- [Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation](https://mlsanity.com/paper/2606.23743/) (2026)
- [cuPilot: A Strategy-Coordinated Multi-agent Framework for CUDA Kernel Evolution](https://mlsanity.com/paper/2512.16465/) (2025)
- [CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe](https://mlsanity.com/paper/2604.01489/) (2026)
- [KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware](https://mlsanity.com/paper/2603.08721/) (2026)
- [CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs](https://mlsanity.com/paper/2605.19269/) (2026)
- [Optimizing PyTorch Inference with LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2511.16964/) (2025)
- [MaxCode: A Max-Reward Reinforcement Learning Framework for Automated Code Optimization](https://mlsanity.com/paper/2601.05475/) (2026)
- [Fine-Tuning GPT-5 for GPU Kernel Generation](https://mlsanity.com/paper/2602.11000/) (2026)
- [Making LLMs Optimize Multi-Scenario CUDA Kernels Like Experts](https://mlsanity.com/paper/2603.07169/) (2026)
- [KForge: Program Synthesis for Diverse AI Hardware Accelerators](https://mlsanity.com/paper/2511.13274/) (2025)
- [GPU Kernel Optimization Beyond Full Builds: An LLM Framework with Minimal Executable Programs](https://mlsanity.com/paper/2512.22147/) (2025)
- [OptiML: An End-to-End Framework for Program Synthesis and CUDA Kernel Optimization](https://mlsanity.com/paper/2602.12305/) (2026)
- [daVinci-kernel: Co-Evolving Skill Selection, Summarization, and Utilization via RL for GPU Kernel Optimization](https://mlsanity.com/paper/2606.16497/) (2026)
- [CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels](https://mlsanity.com/paper/2605.05023/) (2026)
- [HTAM: Hierarchical Transition-Attended Memory for Operator Optimization](https://mlsanity.com/paper/2605.29734/) (2026)
- [Evolution of Kernels: Automated RISC-V Kernel Optimization with Large Language Models](https://mlsanity.com/paper/2509.14265/) (2025)
- [From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation](https://mlsanity.com/paper/2606.16231/) (2026)
- [CRINN: Contrastive Reinforcement Learning for Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2508.02091/) (2025)
- [KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators](https://mlsanity.com/paper/2606.02963/) (2026)
- [KernelBrain: Coarse-to-Fine, Budget-Aware Search for Agentic GPU Kernel Optimization](https://mlsanity.com/paper/2608.02611/) (2026)
- [Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?](https://mlsanity.com/paper/2608.03983/) (2026)
- [HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization](https://mlsanity.com/paper/2608.21157/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated strong capabilities in general-purpose code generation. However, generating the code which is deeply hardware-specific, architecture-aware, and performance-critical, especially for massively parallel GPUs, remains a complex challenge. In this work, we explore the use of LLMs for the automated generation and optimization of CUDA programs, with the goal of producing high-performance GPU kernels that fully exploit the underlying hardware. To address this challenge, we propose a novel framework called Feature Search and Reinforcement (FSR). FSR jointly optimizes compilation and functional correctness, as well as the runtime performance, which are validated through extensive and diverse test cases, and measured by actual kernel execution latency on the target GPU, respectively. This approach enables LLMs not only to generate syntactically and semantically correct CUDA code but also to iteratively refine it for efficiency, tailored to the characteristics of the GPU architecture. We evaluate FSR on representative CUDA kernels, covering AI workloads and computational intensive algorithms. Our results show that LLMs augmented with FSR consistently guarantee correctness rates. Meanwhile, the automatically generated kernels can outperform general human-written code by a factor of up to 179× in execution speeds. These findings highlight the potential of combining LLMs with performance reinforcement to automate GPU programming for hardware-specific, architecture-sensitive, and performance-critical applications.