# HIERA: Workload-Aware Planning Across Implementation Spaces for GPU Kernel Optimization

arXiv: https://arxiv.org/abs/2608.21157 · HTML: https://mlsanity.com/paper/2608.21157/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 14 of 14)

- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [Kevin: Multi-Turn RL for Generating CUDA Kernels](https://mlsanity.com/paper/2507.11948/) (2025)
- [CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization](https://mlsanity.com/paper/2511.01884/) (2025)
- [TileLang: A Composable Tiled Programming Model for AI Systems](https://mlsanity.com/paper/2504.17577/) (2025)
- [Mirage: A Multi-Level Superoptimizer for Tensor Programs](https://mlsanity.com/paper/2405.05751/) (2024)
- [TensorIR: An Abstraction for Automatic Tensorized Program Optimization](https://mlsanity.com/paper/2207.04296/) (2022)
- [ThunderKittens: Simple, Fast, and Adorable AI Kernels](https://mlsanity.com/paper/2410.20399/) (2024)
- [CUDA-LLM: LLMs Can Write Efficient CUDA Kernels](https://mlsanity.com/paper/2506.09092/) (2025)
- [Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks](https://mlsanity.com/paper/2507.23194/) (2025)
- [UNIT: Unifying Tensorized Instruction Compilation](https://mlsanity.com/paper/2101.08458/) (2021)
- [KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?](https://mlsanity.com/paper/2607.16241/) (2026)
- [CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe](https://mlsanity.com/paper/2604.01489/) (2026)
- [The Correctness Illusion in LLM-Generated GPU Kernels](https://mlsanity.com/paper/2606.20128/) (2026)

## Abstract (arXiv)

High-performance GPU kernels underpin modern deep learning and scientific computing. As workloads become increasingly diverse and GPU hardware evolves rapidly, developing efficient methods for automated GPU kernel generation and optimization has become increasingly important. Existing LLM-based methods typically optimize within a fixed implementation space, limiting either optimization flexibility or search efficiency. We propose textscHIERA, a hierarchical search-space planning framework for GPU kernel optimization. textscHIERA constructs contract-augmented task specifications, selects an appropriate implementation space across PyTorch operators, CUDA libraries, and custom CUDA kernels, and uses profiling feedback and expert knowledge to guide structured iterative refinement. Experiments on KernelBench across multiple various workload levels and base LLMs show that textscHIERA delivers stronger overall implementation validity, sample efficiency, and optimization performance than existing training-free methods, while remaining competitive with the training-based CUDA-L1 without additional model training. A case study on a specialized stencil operator from scientific computing further achieves a (1.53×) speedup over cuDNN, demonstrating the potentiality of the general framework beyond standard machine-learning workloads.