# ChipSeek: Optimizing Verilog Generation via EDA-Integrated Reinforcement Learning

arXiv: https://arxiv.org/abs/2507.04736 · HTML: https://mlsanity.com/paper/2507.04736/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 7 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 15 of 15)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023)
- [BetterV: Controlled Verilog Generation with Discriminative Guidance](https://mlsanity.com/paper/2402.03375/) (2024)
- [VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool](https://mlsanity.com/paper/2408.08927/) (2024)
- [Make Every Move Count: LLM-based High-Quality RTL Code Generation Using MCTS](https://mlsanity.com/paper/2402.03289/) (2024)
- [CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair](https://mlsanity.com/paper/2409.12993/) (2024)
- [Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework](https://mlsanity.com/paper/2403.11202/) (2024)
- [AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs](https://mlsanity.com/paper/2407.18333/) (2024)
- [RTLRewriter: Methodologies for Large Models aided RTL Code Optimization](https://mlsanity.com/paper/2409.11414/) (2024)
- [Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systems](https://mlsanity.com/paper/2506.13905/) (2025)
- [SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning](https://mlsanity.com/paper/2504.10369/) (2025)
- [ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model](https://mlsanity.com/paper/2504.14560/) (2025)
- [Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation](https://mlsanity.com/paper/2407.08473/) (2024)

## Built on by (most cited first, 5 of 5)

- [VeriAgent: A Tool-Integrated Multi-Agent System with Evolving Memory for PPA-Aware RTL Code Generation](https://mlsanity.com/paper/2603.17613/) (2026)
- [RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning](https://mlsanity.com/paper/2603.27630/) (2026)
- [Alpha-RTL: Test-Time Training for RTL Hardware Optimization](https://mlsanity.com/paper/2606.05253/) (2026)
- [Agentic Electronic Design Automation: A Handoff Perspective](https://mlsanity.com/paper/2606.19795/) (2026)
- [ChipMEM: Verification-Grounded Memory for EDA Agents](https://mlsanity.com/paper/2609.27067/) (2026)

## Abstract (arXiv)

Large Language Models have emerged as powerful tools for automating Register-Transfer Level (RTL) code generation, yet they face critical limitations: existing approaches typically fail to simultaneously optimize functional correctness and hardware efficiency metrics such as Power, Performance, and Area (PPA). Methods relying on supervised fine-tuning commonly produce functionally correct but suboptimal designs due to the lack of inherent mechanisms for learning hardware optimization principles. Conversely, external post-processing techniques aiming to refine PPA performance after generation often suffer from inefficiency and do not improve the LLMs' intrinsic capabilities. To overcome these challenges, we propose ChipSeek, a novel hierarchical reward based reinforcement learning framework designed to encourage LLMs to generate RTL code that is both functionally correct and optimized for PPA metrics. Our approach integrates direct feedback from EDA simulators and synthesis tools into a hierarchical reward mechanism, facilitating a nuanced understanding of hardware design trade-offs. Through Curriculum-Guided Dynamic Policy Optimization (CDPO), ChipSeek enhances the LLM's ability to generate high-quality, optimized RTL code. Evaluations on standard benchmarks demonstrate ChipSeek's superior performance, achieving state-of-the-art functional correctness and PPA performance. Furthermore, it excels in specific optimization tasks, consistently yielding highly efficient designs when individually targeting fine-grained optimization goals such as power, delay, and area. The artifact is open-source in https://github.com/rong-hash/chipseek.