# CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization

arXiv: https://arxiv.org/abs/2511.01884 · HTML: https://mlsanity.com/paper/2511.01884/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 22 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [A Survey on Code Generation with LLM-based Agents](https://mlsanity.com/paper/2508.00083/) (2025)
- [Kevin: Multi-Turn RL for Generating CUDA Kernels](https://mlsanity.com/paper/2507.11948/) (2025)
- [CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning](https://mlsanity.com/paper/2507.14111/) (2025)
- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization](https://mlsanity.com/paper/2509.14279/) (2025)
- [LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving](https://mlsanity.com/paper/2509.01229/) (2025)
- [InfantAgent-Next: A Multimodal Generalist Agent for Automated Computer Interaction](https://mlsanity.com/paper/2505.10887/) (2025)

## Built on by (most cited first, 30 of 45)

- [CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation](https://mlsanity.com/paper/2602.24286/) (2026)
- [K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model](https://mlsanity.com/paper/2602.19128/) (2026)
- [AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search](https://mlsanity.com/paper/2603.21331/) (2026)
- [KernelFoundry: Hardware-aware evolutionary GPU kernel optimization](https://mlsanity.com/paper/2603.12440/) (2026)
- [KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning](https://mlsanity.com/paper/2602.14293/) (2026)
- [Towards Automated Kernel Generation in the Era of LLMs](https://mlsanity.com/paper/2601.15727/) (2026)
- [KernelSkill: A Multi-Agent Framework for GPU Kernel Optimization](https://mlsanity.com/paper/2603.10085/) (2026)
- [AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement](https://mlsanity.com/paper/2608.20318/) (2026)
- [Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation](https://mlsanity.com/paper/2606.23743/) (2026)
- [StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning](https://mlsanity.com/paper/2603.02637/) (2026)
- [Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization](https://mlsanity.com/paper/2603.28342/) (2026)
- [CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe](https://mlsanity.com/paper/2604.01489/) (2026)
- [Making LLMs Optimize Multi-Scenario CUDA Kernels Like Experts](https://mlsanity.com/paper/2603.07169/) (2026)
- [Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization](https://mlsanity.com/paper/2606.26453/) (2026)
- [DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels](https://mlsanity.com/paper/2602.11715/) (2026)
- [DRTriton: Large-Scale Synthetic Data Driven Reinforcement Learning for Triton Kernel Generation](https://mlsanity.com/paper/2603.21465/) (2026)
- [AscendOptimizer: Episodic Agent for Ascend NPU Operator Optimization](https://mlsanity.com/paper/2603.23566/) (2026)
- [FastKernels: Benchmarking GPU Kernel Generation in Production](https://mlsanity.com/paper/2605.23215/) (2026)
- [PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization](https://mlsanity.com/paper/2607.19653/) (2026)
- [KernelBand: Steering LLM-based Kernel Optimization via Hardware-Aware Multi-Armed Bandits](https://mlsanity.com/paper/2511.18868/) (2025)
- [CUCo: An Agentic Framework for Compute and Communication Co-design](https://mlsanity.com/paper/2603.02376/) (2026)
- [CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels](https://mlsanity.com/paper/2605.05023/) (2026)
- [HTAM: Hierarchical Transition-Attended Memory for Operator Optimization](https://mlsanity.com/paper/2605.29734/) (2026)
- [MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU](https://mlsanity.com/paper/2606.04847/) (2026)
- [MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?](https://mlsanity.com/paper/2603.11935/) (2026)
- [CUDAHercules: Benchmarking Hardware-Aware Expert-level CUDA Optimization for LLMs](https://mlsanity.com/paper/2605.08467/) (2026)
- [Hawk: Harnessing Hardware-Aware Knowledge for High-Performance NPU Kernel Generation](https://mlsanity.com/paper/2607.01590/) (2026)
- [FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications](https://mlsanity.com/paper/2607.18171/) (2026)
- [Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels](https://mlsanity.com/paper/2607.24762/) (2026)
- [Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariants](https://mlsanity.com/paper/2604.18616/) (2026)

## Abstract (arXiv)

Developing efficient CUDA kernels is increasingly critical for AI applications such as large-scale LLM training. However, manual kernel design is both costly and time-consuming, motivating automatic approaches that leverage LLMs for code generation. Existing methods for automatic kernel generation, however, often produce low-efficiency kernels, incur high computational overhead, and fail to generalize across settings. In this work, we propose CudaForge, a training-free multi-agent workflow for CUDA kernel generation and optimization. Our workflow is inspired by the iterative workflow of human experts, which contains steps such as developing initial kernels, testing correctness, analyzing hardware feedback, and iterative improvement. More specifically, CudaForge employs two LLM agents: a Coder and a Judge, that iteratively generate, correct, and optimize CUDA kernels, while integrating hardware feedback such as Nsight Compute (NCU) metrics. In extensive evaluations, we show that CudaForge, by leveraging base models like OpenAI-o3, achieves 97.6% correctness of generated kernels and an average 1.68× speedup over PyTorch baselines, substantially surpassing state-of-the-art models including OpenAI-o3 and Kevin on KernelBench.Beyond accuracy and speed, CudaForge demonstrates strong generalization across GPUs (A100, RTX 6000, 4090, 3090) and base models (OpenAI-o3, GPT-5, gpt-oss-120B, Claude-Sonnet-4, QwQ-32B), while maintaining high efficiency. In particular, generating an optimized kernel takes about 26.5 minutes on one RTX6000 and incurs about  0.3 API cost, which is significantly cheaper than existing agentic work that costs 6 H100 hours and  5 API cost per kernel. Our results highlight that multi-agent, training-free workflows can enable cost-effective, generalizable, and high-performance CUDA kernel optimization. Code available at https://github.com/OptimAI-Lab/CudaForge