mlsanityData to 7 Oct 2026

PerfReasoning: How Well Do LLMs Reason on Hardware Performance?

arXiv 2609.04476 ↗First posted 3 Sep 2026cs.AI cs.PFcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Performance modeling is central to hardware design and software optimization, yet constructing these models requires structured reasoning about computation, data reuse, storage, and movement. We introduce PerfReasoning, a benchmark that evaluates LLMs both as direct performance reasoners and as generators of analytical performance-model code. Given workload, architecture, and mapping specifications, models compare mappings and predict off-chip traffic and buffer requirements. The strongest closed-source models exceed 90% on reasoning-based Q&A, and the best open-weight model reaches 82.4%. However, model construction is substantially harder: while GPT-5.6 Sol exceeds 80% pass rate, all other model configurations average below 45% and vary markedly across runs. Task-specific RL raises a 4B model's mapping-reasoning accuracy by 15.7 points, whereas feedback-free multi-round self-revision prompting is not reliably effective. PerfReasoning exposes the gap between plausible architectural reasoning and reliable performance-model construction. We will publicly release the benchmark to support reproducible evaluation and track future progress.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 25 of its 34 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

25 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture24 Oct

Other references · 24

  1. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
  2. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  3. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  4. 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 143
  5. 2023ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scalecited by 47
  6. 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verificationcited by 41
  7. 2025MultiKernelBench: A Multi-Platform Benchmark for Kernel Generationcited by 35
  8. 2025OpenLLM-RTL: Open Dataset and Benchmark for LLM-Aided Design RTL Generationcited by 30
  9. 2021Data-Driven Offline Optimization For Architecting Hardware Acceleratorscited by 29
  10. 2020DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogatescited by 14
  11. 2022The Championship Simulator: Architectural Simulation for Education and Competitioncited by 14
  12. 2020The gem5 Simulator: Version 20.0+cited by 13
12 more
  1. 2026SOL-ExecBench: Speed-of-Light Benchmarking for Real-World GPU Kernels Against Hardware Limitscited by 13
  2. 2025HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Taskscited by 10
  3. 2026KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernelscited by 10
  4. 2025SCALE-Sim v3: A modular cycle-accurate systolic accelerator simulator for end-to-end system analysiscited by 8
  5. 2026ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Designcited by 7
  6. 2023ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Designcited by 6
  7. 2021AIRCHITECT: Learning Custom Architecture Design and Mapping Spacecited by 4
  8. 2026Computer Architecture's AlphaZero Moment: Automated Discovery in an Encircled Worldcited by 4
  9. 2026PDAGENT-BENCH: Characterizing, Grounding, and Architecting LLM/VLM Agents for VLSI Physical Designcited by 3
  10. 2026Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generationcited by 2
  11. 2021Q-error Bounds of Random Uniform Sampling for Cardinality Estimationcited by 1
  12. 2026ArchEval: Measuring AI Agents as Computer Architectscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Counting Without Running: Evaluating LLMs' Reasoning About Code Complexity
  2. 2025RAPID-LLM: Resilience-Aware Performance analysis of Infrastructure for Distributed LLM Training and Inference ↗
  3. 2026Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
  4. 2026SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving
  5. 2026LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure
  6. 2025Towards LLM-based Root Cause Analysis of Hardware Design Failures
  7. 2026Beyond Accuracy: Unveiling Inefficiency Patterns in Tool-Integrated Reasoning
  8. 2026SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops