# Exploring the Agentic Frontier of Verilog Code Generation

arXiv: https://arxiv.org/abs/2603.19347 · HTML: https://mlsanity.com/paper/2603.19347/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 24 of 24)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [VerilogEval: Evaluating Large Language Models for Verilog Code Generation](https://mlsanity.com/paper/2309.07544/) (2023)
- [RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model](https://mlsanity.com/paper/2308.05345/) (2023)
- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://mlsanity.com/paper/2506.14074/) (2025)
- [VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification](https://mlsanity.com/paper/2505.20302/) (2025)
- [ACE-RTL: When Agentic Context Evolution Meets RTL-Specialized LLMs](https://mlsanity.com/paper/2602.10218/) (2026)
- [Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead](https://mlsanity.com/paper/2512.00020/) (2025)
- [A New Benchmark for the Appropriate Evaluation of RTL Code Optimization](https://mlsanity.com/paper/2601.01765/) (2026)
- [LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation](https://mlsanity.com/paper/2602.16953/) (2026)
- [NotSoTiny: A Large, Living Benchmark for RTL Code Generation](https://mlsanity.com/paper/2512.20823/) (2025)
- [GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation](https://mlsanity.com/paper/2601.07593/) (2026)
- [SiliconMind-V1: Multi-Agent Distillation and Debug-Reasoning Workflows for Verilog Code Generation](https://mlsanity.com/paper/2603.08719/) (2026)
- [Assessing Large Language Models in Generating RTL Design Specifications](https://mlsanity.com/paper/2512.00045/) (2025)
- [Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation](https://mlsanity.com/paper/2601.19941/) (2026)
- [LLM-FSM: Scaling Large Language Models for Finite-State Reasoning in RTL Code Generation](https://mlsanity.com/paper/2602.07032/) (2026)
- [SpecLoop: An Agentic RTL-to-Specification Framework with Formal Verification Feedback Loop](https://mlsanity.com/paper/2603.02895/) (2026)
- [Multimodal Chip Physical Design Engineer Assistant](https://mlsanity.com/paper/2510.15872/) (2025)
- [QiMeng-ChipV-RTL: Exploiting Information Locality for IP-level Verilog Generation](https://mlsanity.com/paper/2602.00704/) (2026)
- [Saarthi for AGI: Towards Domain-Specific General Intelligence for Formal Verification](https://mlsanity.com/paper/2603.03175/) (2026)

## Built on by (most cited first, 1 of 1)

- [BEHAVE: Functional Behavior Modeling Enables Self-Improving Agents for Hardware Design and Verification](https://mlsanity.com/paper/2609.34785/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have made rapid advancements in code generation for popular languages such as Python and C++. Many of these recent gains can be attributed to the use of ``agents'' that wrap domain-relevant tools alongside LLMs. Hardware design languages such as Verilog have also seen improved code generation in recent years, but the impact of agentic frameworks on Verilog code generation tasks remains unclear. In this work, we present the first systematic evaluation of agentic LLMs for Verilog generation, using the recently introduced CVDP benchmark. We also introduce several open-source hardware design agent harnesses, providing a model-agnostic baseline for future work. Through controlled experiments across frontier models, we study how structured prompting and tool design affect performance, analyze agent failure modes and tool usage patterns, compare open-source and closed-source models, and provide qualitative examples of successful and failed agent runs. Our results show that naive agentic wrapping around frontier models can degrade performance (relative to standard forward passes with optimized prompts), but that structured harnesses meaningfully match and in some cases exceed non-agentic baselines. We find that the performance gap between open and closed source models is driven by both higher crash rates and weaker tool output interpretation. Our exploration illuminates the path towards designing special-purpose agents for verilog generation in the future.