# AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization

arXiv: https://arxiv.org/abs/2511.15915 · HTML: https://mlsanity.com/paper/2511.15915/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 35 of its 63 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 30 of 35)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://mlsanity.com/paper/2407.08608/) (2024)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving](https://mlsanity.com/paper/2501.01005/) (2025)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [Kevin: Multi-Turn RL for Generating CUDA Kernels](https://mlsanity.com/paper/2507.11948/) (2025)
- [SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience](https://mlsanity.com/paper/2508.04700/) (2025)
- [CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning](https://mlsanity.com/paper/2507.14111/) (2025)
- [Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization](https://mlsanity.com/paper/2509.14279/) (2025)
- [Mirage: A Multi-Level Superoptimizer for Tensor Programs](https://mlsanity.com/paper/2405.05751/) (2024)
- [Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures](https://mlsanity.com/paper/2505.09343/) (2025)
- [MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation](https://mlsanity.com/paper/2507.17773/) (2025)
- [ThunderKittens: Simple, Fast, and Adorable AI Kernels](https://mlsanity.com/paper/2410.20399/) (2024)
- [TritonRL: Training LLMs to Think and Code Triton Without Cheating](https://mlsanity.com/paper/2510.17891/) (2025)
- [From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation](https://mlsanity.com/paper/2502.00330/) (2025)
- [SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts](https://mlsanity.com/paper/2405.07518/) (2024)

## Built on by (most cited first, 23 of 23)

- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [Understanding the Challenges in Iterative Generative Optimization with LLMs](https://mlsanity.com/paper/2603.23994/) (2026)
- [CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution](https://mlsanity.com/paper/2608.12629/) (2026)
- [AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation](https://mlsanity.com/paper/2604.16625/) (2026)
- [PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization](https://mlsanity.com/paper/2607.19653/) (2026)
- [From Human Guidance to Autonomy: Agent Skill System for End-to-End LLM Deployment on Spatial NPUs](https://mlsanity.com/paper/2606.07586/) (2026)
- [NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation](https://mlsanity.com/paper/2607.04395/) (2026)
- [Architectural Implications of Agentic AI Workflows](https://mlsanity.com/paper/2608.04458/) (2026)
- [Surprisal-Guided Selection: Compute-Optimal Test-Time Strategies for Execution-Grounded Code Generation](https://mlsanity.com/paper/2602.07670/) (2026)
- [RuleFlow : Generating Reusable Program Optimizations with LLMs](https://mlsanity.com/paper/2602.09051/) (2026)
- [MKEvolve: A Modular Multi-Agent Framework for Kernel Code Generation](https://mlsanity.com/paper/2607.20501/) (2026)
- [Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages](https://mlsanity.com/paper/2605.28213/) (2026)
- [Arbor: Tree Search as a Cognition Layer for Autonomous Agents](https://mlsanity.com/paper/2606.12563/) (2026)
- [IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurations](https://mlsanity.com/paper/2607.26075/) (2026)
- [KernelArc: A Multi-Agent Framework for GPU Kernel Optimization](https://mlsanity.com/paper/2608.17071/) (2026)
- [PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTX](https://mlsanity.com/paper/2608.17379/) (2026)
- [From PyTorch to the NPU: LLM-Agent-Driven Model Conversion Across Heterogeneous Inference Runtimes](https://mlsanity.com/paper/2609.27249/) (2026)
- [KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization](https://mlsanity.com/paper/2609.30059/) (2026)
- [Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edge](https://mlsanity.com/paper/2609.35110/) (2026)
- [From Experience to Expertise: Adoption-Aware Memory Learning for Data-Scarce NPU Kernel Synthesis](https://mlsanity.com/paper/2609.35568/) (2026)
- [GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation](https://mlsanity.com/paper/2609.35639/) (2026)
- [D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?](https://mlsanity.com/paper/2610.03226/) (2026)
- [ML Engineering Agents - A Data Management Perspective](https://mlsanity.com/paper/2610.16919/) (2026)

## Abstract (arXiv)

We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimizes kernels for emerging AI acclerators, eliminating the need for expert-provided hardware-specific optimization knowledge. AccelOpt explores the kernel optimization space through iterative generation, informed by an optimization memory that curates experiences and insights from previously encountered slow-fast kernel pairs. We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world LLM workloads to evaluate the effectiveness of AccelOpt. Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from 49% to 61% on Trainium 1 and from 45% to 59% on Trainium 2 for NKIBench kernels. Moreover, AccelOpt is highly cost-effective: using open-source models, it matches the kernel improvements of Claude Sonnet 4 while being 26× cheaper. The code is open-sourced at https://github.com/zhang677/AccelOpt.