# SwizzlePerf: Hardware-Aware LLMs for GPU Kernel Performance Optimization

arXiv: https://arxiv.org/abs/2508.20258 · HTML: https://mlsanity.com/paper/2508.20258/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://mlsanity.com/paper/2212.14024/) (2022)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Flex Attention: A Programming Model for Generating Optimized Attention Kernels](https://mlsanity.com/paper/2412.05496/) (2024)
- [Learning to Optimize Tensor Programs](https://mlsanity.com/paper/1805.08166/) (2018)
- [Learning Performance-Improving Code Edits](https://mlsanity.com/paper/2302.07867/) (2023)
- [Kevin: Multi-Turn RL for Generating CUDA Kernels](https://mlsanity.com/paper/2507.11948/) (2025)
- [CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning](https://mlsanity.com/paper/2507.14111/) (2025)
- [Multi-Head Attention: Collaborate Instead of Concatenate](https://mlsanity.com/paper/2006.16362/) (2020)
- [GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization](https://mlsanity.com/paper/2506.20807/) (2025)
- [Lean Attention: Hardware-Aware Scalable Attention Mechanism for the Decode-Phase of Transformers](https://mlsanity.com/paper/2405.10480/) (2024)
- [QuArch: A Question-Answering Dataset for AI Agents in Computer Architecture](https://mlsanity.com/paper/2501.01892/) (2025)

## Built on by (most cited first, 11 of 11)

- [CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning](https://mlsanity.com/paper/2512.02551/) (2025)
- [KernelFoundry: Hardware-aware evolutionary GPU kernel optimization](https://mlsanity.com/paper/2603.12440/) (2026)
- [PRAGMA: A Profiling-Reasoned Multi-Agent Framework for Automatic Kernel Optimization](https://mlsanity.com/paper/2511.06345/) (2025)
- [Towards Automated Kernel Generation in the Era of LLMs](https://mlsanity.com/paper/2601.15727/) (2026)
- [cuPilot: A Strategy-Coordinated Multi-agent Framework for CUDA Kernel Evolution](https://mlsanity.com/paper/2512.16465/) (2025)
- [QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture](https://mlsanity.com/paper/2510.22087/) (2025)
- [KernelBand: Steering LLM-based Kernel Optimization via Hardware-Aware Multi-Armed Bandits](https://mlsanity.com/paper/2511.18868/) (2025)
- [JAXBench: Benchmarking Autonomous TPU Kernel Optimization](https://mlsanity.com/paper/2607.20466/) (2026)
- [Integrating Performance Tools in Model Reasoning for GPU Kernel Optimization](https://mlsanity.com/paper/2510.17158/) (2025)
- [Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation](https://mlsanity.com/paper/2605.26720/) (2026)
- [Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts](https://mlsanity.com/paper/2605.30359/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have shown progress in GPU kernel performance engineering using inefficient search-based methods that optimize around runtime. Any existing approach lacks a key characteristic that human performance engineers rely on for near-optimal utilization -- hardware-awareness. By leveraging the workload's specific memory access patterns, architecture specifications, filtered profiling logs, and reflections on historical performance, we can make software-level optimizations that are tailored to the underlying hardware. SwizzlePerf automatically generates spatial optimizations for GPU kernels on disaggregated architectures by giving LLMs explicit hardware-awareness. For a GEMM kernel, SwizzlePerf takes less than 5 minutes to generate the same hardware-specific optimal swizzling pattern that took expert performance engineers 2 weeks to find. On a suite of 10 diverse ML and Science kernels, SwizzlePerf can generate swizzling patterns for 9 of the kernels that achieve up to a 2.06x speedup and 70% improvement in L2 hit rate. This work is the first of many steps toward systematically creating hardware-aware LLM performance engineering agents.