# MetRex: A Benchmark for Verilog Code Metric Reasoning Using LLMs

arXiv: https://arxiv.org/abs/2411.03471 · HTML: https://mlsanity.com/paper/2411.03471/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 18 of 18)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2401.03065/) (2024)
- [GraphGPT: Graph Instruction Tuning for Large Language Models](https://mlsanity.com/paper/2310.13023/) (2023)
- [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023)
- [Talk like a Graph: Encoding Graphs for Large Language Models](https://mlsanity.com/paper/2310.04560/) (2023)
- [RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model](https://mlsanity.com/paper/2308.05345/) (2023)
- [ChatEDA: A Large Language Model Powered Autonomous Agent for EDA](https://mlsanity.com/paper/2308.10204/) (2023)
- [Benchmarking Large Language Models for Automated Verilog RTL Code Generation](https://mlsanity.com/paper/2212.11140/) (2022)
- [InferFix: End-to-End Program Repair with LLMs](https://mlsanity.com/paper/2303.07263/) (2023)
- [RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models](https://mlsanity.com/paper/2311.16543/) (2023)
- [RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution](https://mlsanity.com/paper/2312.08617/) (2023)
- [GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models](https://mlsanity.com/paper/2309.10730/) (2023)
- [DAVE: Deriving Automatically Verilog from English](https://mlsanity.com/paper/2009.01026/) (2020)
- [Meta Large Language Model Compiler: Foundation Models of Compiler Optimization](https://mlsanity.com/paper/2407.02524/) (2024)
- [RTLRewriter: Methodologies for Large Models aided RTL Code Optimization](https://mlsanity.com/paper/2409.11414/) (2024)
- [MasterRTL: A Pre-Synthesis PPA Estimation Framework for Any RTL Design](https://mlsanity.com/paper/2311.08441/) (2023)
- [PANDA: Architecture-Level Power Evaluation by Unifying Analytical and Machine Learning Solutions](https://mlsanity.com/paper/2312.08994/) (2023)

## Built on by (most cited first, 7 of 7)

- [VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification](https://mlsanity.com/paper/2505.20302/) (2025)
- [Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead](https://mlsanity.com/paper/2512.00020/) (2025)
- [ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness](https://mlsanity.com/paper/2503.08823/) (2025)
- [Automated Multi-Agent Workflows for RTL Design](https://mlsanity.com/paper/2509.20182/) (2025)
- [Pluto: A Benchmark for Evaluating Efficiency of LLM-generated Hardware Code](https://mlsanity.com/paper/2510.14756/) (2025)
- [RocketPPA: Code-Level Power, Performance, and Area Prediction via LLM and Mixture of Experts](https://mlsanity.com/paper/2503.21971/) (2025)
- [Benchmarking LLMs for Verilog Design Flows](https://mlsanity.com/paper/2607.22759/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have been applied to various hardware design tasks, including Verilog code generation, EDA tool scripting, and RTL bug fixing. Despite this extensive exploration, LLMs are yet to be used for the task of post-synthesis metric reasoning and estimation of HDL designs. In this paper, we assess the ability of LLMs to reason about post-synthesis metrics of Verilog designs. We introduce MetRex, a large-scale dataset comprising 25,868 Verilog HDL designs and their corresponding post-synthesis metrics, namely area, delay, and static power. MetRex incorporates a Chain of Thought (CoT) template to enhance LLMs' reasoning about these metrics. Extensive experiments show that Supervised Fine-Tuning (SFT) boosts the LLM's reasoning capabilities on average by 37.0%, 25.3%, and 25.7% on the area, delay, and static power, respectively. While SFT improves performance on our benchmark, it remains far from achieving optimal results, especially on complex problems. Comparing to state-of-the-art regression models, our approach delivers accurate post-synthesis predictions for 17.4% more designs (within a 5% error margin), in addition to offering a 1.7x speedup by eliminating the need for pre-processing. This work lays the groundwork for advancing LLM-based Verilog code metric reasoning.