# PerfReasoning: How Well Do LLMs Reason on Hardware Performance?

arXiv: https://arxiv.org/abs/2609.04476 · HTML: https://mlsanity.com/paper/2609.04476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 25 of 25)

- [VerilogEval: Evaluating Large Language Models for Verilog Code Generation](https://mlsanity.com/paper/2309.07544/) (2023)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [Ansor: Generating High-Performance Tensor Programs for Deep Learning](https://mlsanity.com/paper/2006.06762/) (2020)
- [RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model](https://mlsanity.com/paper/2308.05345/) (2023)
- [ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale](https://mlsanity.com/paper/2303.14006/) (2023)
- [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://mlsanity.com/paper/2506.14074/) (2025)
- [MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation](https://mlsanity.com/paper/2507.17773/) (2025)
- [OpenLLM-RTL: Open Dataset and Benchmark for LLM-Aided Design RTL Generation](https://mlsanity.com/paper/2503.15112/) (2025)
- [Data-Driven Offline Optimization For Architecting Hardware Accelerators](https://mlsanity.com/paper/2110.11346/) (2021)
- [SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limits](https://mlsanity.com/paper/2603.19173/) (2026)
- [The Championship Simulator: Architectural Simulation for Education and Competition](https://mlsanity.com/paper/2210.14324/) (2022)
- [The gem5 Simulator: Version 20.0+](https://mlsanity.com/paper/2007.03152/) (2020)
- [DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogates](https://mlsanity.com/paper/2010.04017/) (2020)
- [KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels](https://mlsanity.com/paper/2605.04956/) (2026)
- [HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks](https://mlsanity.com/paper/2504.12268/) (2025)
- [SCALE-Sim v3: A modular cycle-accurate systolic accelerator simulator for end-to-end system analysis](https://mlsanity.com/paper/2504.15377/) (2025)
- [ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design](https://mlsanity.com/paper/2601.21448/) (2026)
- [ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design](https://mlsanity.com/paper/2306.08888/) (2023)
- [Computer Architecture's AlphaZero Moment: Automated Discovery in an Encircled World](https://mlsanity.com/paper/2604.03312/) (2026)
- [AIRCHITECT: Learning Custom Architecture Design and Mapping Space](https://mlsanity.com/paper/2108.08295/) (2021)
- [PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Design](https://mlsanity.com/paper/2606.17253/) (2026)
- [QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture](https://mlsanity.com/paper/2510.22087/) (2025)
- [Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation](https://mlsanity.com/paper/2601.19941/) (2026)
- [Q-error Bounds of Random Uniform Sampling for Cardinality Estimation](https://mlsanity.com/paper/2108.02715/) (2021)
- [ArchEval: Measuring AI Agents as Computer Architects](https://mlsanity.com/paper/2607.03601/) (2026)

## Abstract (arXiv)

Performance modeling is central to hardware design and software optimization, yet constructing these models requires structured reasoning about computation, data reuse, storage, and movement. We introduce PerfReasoning, a benchmark that evaluates LLMs both as direct performance reasoners and as generators of analytical performance-model code. Given workload, architecture, and mapping specifications, models compare mappings and predict off-chip traffic and buffer requirements. The strongest closed-source models exceed 90% on reasoning-based Q&A, and the best open-weight model reaches 82.4%. However, model construction is substantially harder: while GPT-5.6 Sol exceeds 80% pass rate, all other model configurations average below 45% and vary markedly across runs. Task-specific RL raises a 4B model's mapping-reasoning accuracy by 15.7 points, whereas feedback-free multi-round self-revision prompting is not reliably effective. PerfReasoning exposes the gap between plausible architectural reasoning and reliable performance-model construction. We will publicly release the benchmark to support reproducible evaluation and track future progress.