mlsanityData to 7 Oct 2026

AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization

arXiv 2511.15915 ↗First posted 19 Nov 2025cs.LG cs.CLcited by 21

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimizes kernels for emerging AI acclerators, eliminating the need for expert-provided hardware-specific optimization knowledge. AccelOpt explores the kernel optimization space through iterative generation, informed by an optimization memory that curates experiences and insights from previously encountered slow-fast kernel pairs. We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world LLM workloads to evaluate the effectiveness of AccelOpt. Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from 49% to 61% on Trainium 1 and from 45% to 59% on Trainium 2 for NKIBench kernels. Moreover, AccelOpt is highly cost-effective: using open-source models, it matches the kernel improvements of Claude Sonnet 4 while being 26× cheaper. The code is open-sourced at https://github.com/zhang677/AccelOpt.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 63 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

35 of 63 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization16 Sep
  2. 2025AlphaEvolve: A coding agent for scientific and algorithmic discovery16 Jun
  3. 2025Adaptive Self-improvement LLM Agentic System for ML Library Development4 Feb

Other references · 32

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  7. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  8. 2023The Falcon Series of Open Language Modelscited by 526
  9. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  10. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  11. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  12. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
20 more
  1. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  2. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 248
  3. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  4. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  5. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  6. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  7. 2025Kevin: Multi-Turn RL for Generating CUDA Kernelscited by 73
  8. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  9. 2025CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningcited by 59
  10. 2024Mirage: A Multi-Level Superoptimizer for Tensor Programscited by 39
  11. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architecturescited by 38
  12. 2025MultiKernelBench: A Multi-Platform Benchmark for Kernel Generationcited by 35
  13. 2024ThunderKittens: Simple, Fast, and Adorable AI Kernelscited by 34
  14. 2025TritonRL: Training LLMs to Think and Code Triton Without Cheatingcited by 24
  15. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 20
  16. 2024SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Expertscited by 18
  17. 2024Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfacescited by 11
  18. 2025Dato: A Task-Based Programming Model for Dataflow Acceleratorscited by 5
  19. 2022Stardust: Compiling Sparse Tensor Algebra to a Reconfigurable Dataflow Architecturecited by 3
  20. 2025TTrace: Lightweight Error Checking and Diagnosis for Distributed Trainingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  2. 2026Understanding the Challenges in Iterative Generative Optimization with LLMscited by 8
  3. 2026AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generationcited by 3
  4. 2026From Human Guidance to Autonomy: Agent Skill System for End-to-End LLM Deployment on Spatial NPUscited by 2
  5. 2026NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generationcited by 2
  6. 2026PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimizationcited by 2
  7. 2026Architectural Implications of Agentic AI Workflowscited by 2
  8. 2026CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolutioncited by 2
  9. 2026Surprisal-Guided Selection: Compute-Optimal Test-Time Strategies for Execution-Grounded Code Generationcited by 1
  10. 2026RuleFlow : Generating Reusable Program Optimizations with LLMscited by 1
  11. 2026MKEvolve: A Modular Multi-Agent Framework for Kernel Code Generationcited by 1
  12. 2026Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineagescited by 0
9 more
  1. 2026Arbor: Tree Search as a Cognition Layer for Autonomous Agentscited by 0
  2. 2026IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurationscited by 0
  3. 2026KernelArc: A Multi-Agent Framework for GPU Kernel Optimizationcited by 0
  4. 2026PTXBench: Benchmarking and Adapting LLMs for GPU Kernel Optimization with Architecture-specific PTXcited by 0
  5. 2026From PyTorch to the NPU: LLM-Agent-Driven Model Conversion Across Heterogeneous Inference Runtimescited by 0
  6. 2026KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimizationcited by 0
  7. 2026Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edgecited by 0
  8. 2026From Experience to Expertise: Adoption-Aware Memory Learning for Data-Scarce NPU Kernel Synthesiscited by 0
  9. 2026GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KernelGenBench: Can LLMs and Agents Write Efficient Kernels Across Operator Sources and Hardware Platforms?
  2. 2026KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware
  3. 2025STARK: Strategic Team of Agents for Refining Kernels
  4. 2026Towards Automated Kernel Generation in the Era of LLMs
  5. 2026LLM4LLM: Bridging Kernel Benchmarks and Real Deployment via Closed-Loop Agentic Optimization
  6. 2026KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators
  7. 2025ASAP: an Agentic Solution to Auto-optimize Performance of Large-Scale LLM Training
  8. 2026AI Hardware Accelerators for Large Language Models: Architectures and the Memory Wall ↗