# GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation arXiv: https://arxiv.org/abs/2601.07593 ยท HTML: https://mlsanity.com/paper/2601.07593/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. - Builds on 31 of its 39 references that are in the map. - First posted on arXiv 12 Jan 2026. ## Placed in - LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/) ## Builds on (most cited first, 30 of 31) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025) - [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022) - [VerilogEval: Evaluating Large Language Models for Verilog Code Generation](https://mlsanity.com/paper/2309.07544/) (2023) - [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023) - [ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models](https://mlsanity.com/paper/2505.24864/) (2025) - [RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model](https://mlsanity.com/paper/2308.05345/) (2023) - [RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models](https://mlsanity.com/paper/2311.16543/) (2023) - [BetterV: Controlled Verilog Generation with Discriminative Guidance](https://mlsanity.com/paper/2402.03375/) (2024) - [VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool](https://mlsanity.com/paper/2408.08927/) (2024) - [AutoChip: Automating HDL Generation Using LLM Feedback](https://mlsanity.com/paper/2311.04887/) (2023) - [RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution](https://mlsanity.com/paper/2312.08617/) (2023) - [OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection](https://mlsanity.com/paper/2407.16237/) (2024) - [DAVE: Deriving Automatically Verilog from English](https://mlsanity.com/paper/2009.01026/) (2020) - [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://mlsanity.com/paper/2506.14074/) (2025) - [AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design](https://mlsanity.com/paper/2407.03891/) (2024) - [Towards LLM-Powered Verilog RTL Assistant: Self-Verification and Self-Correction](https://mlsanity.com/paper/2406.00115/) (2024) - [AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs](https://mlsanity.com/paper/2407.18333/) (2024) - [MAGE: A Multi-Agent Engine for Automated RTL Code Generation](https://mlsanity.com/paper/2412.07822/) (2024) - [HDLdebugger: Streamlining HDL debugging with Large Language Models](https://mlsanity.com/paper/2403.11671/) (2024) - [MEIC: Re-thinking RTL Debug Automation using LLMs](https://mlsanity.com/paper/2405.06840/) (2024) - [CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design](https://mlsanity.com/paper/2411.08510/) (2024) - [LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation](https://mlsanity.com/paper/2310.04535/) (2023) - [DeepRTL: Bridging Verilog Understanding and Generation with a Unified Representation Model](https://mlsanity.com/paper/2502.15832/) (2025) - [ScaleRTL: Scaling LLMs with Reasoning Data and Test-Time Compute for Accurate RTL Code Generation](https://mlsanity.com/paper/2506.05566/) (2025) - [VerilogReader: LLM-Aided Hardware Test Generation](https://mlsanity.com/paper/2406.04373/) (2024) - [AIvril: AI-Driven RTL Generation With Verification In-The-Loop](https://mlsanity.com/paper/2409.11411/) (2024) - [VeriContaminated: Assessing LLM-Driven Verilog Coding for Data Contamination](https://mlsanity.com/paper/2503.13572/) (2025) - [VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction](https://mlsanity.com/paper/2504.19099/) (2025) - [VRank: Enhancing Verilog Code Generation from Large Language Models via Self-Consistency](https://mlsanity.com/paper/2502.00028/) (2025) ## Built on by (most cited first, 4 of 4) - [Exploring the Agentic Frontier of Verilog Code Generation](https://mlsanity.com/paper/2603.19347/) (2026) - [Towards Reliable AI-Assisted Analog Design: Template-Constrained LLM Agents for SAR ADC Generation](https://mlsanity.com/paper/2607.14165/) (2026) - [CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning](https://mlsanity.com/paper/2609.19189/) (2026) - [Trust, but Validate the Instrument: Auditing AI-Generated RTL Verification Plans on Authored Security-Regression Proxies](https://mlsanity.com/paper/2609.19844/) (2026) ## Abstract (arXiv) RTL design often relies heavily on ad-hoc testbench creation early in the design cycle. While large language models (LLMs) show promise for RTL code generation, their ability to reason about hardware specifications and generate targeted test plans remains largely unexplored. We present the first systematic study of LLM reasoning capabilities for RTL verification stimuli generation, establishing a two-stage framework that decomposes test plan generation from testbench execution. Our benchmark reveals that state-of-the-art models, including DeepSeek-R1 and Claude-4.0-Sonnet, achieve only 15.7-21.7% success rates on generating stimuli that pass golden RTL designs. To improve LLM generated stimuli, we develop a comprehensive training methodology combining supervised fine-tuning with a novel reinforcement learning approach, GRPO with State Mutation (GRPO-SMu), which enhances exploration by varying input mutations. Our approach leverages a tree-based branching mutation strategy to construct training data comprising equivalent and mutated trees, moving beyond linear mutation approaches to provide rich learning signals. Training on this curated dataset, our 7B parameter model achieves a 33.3% golden test pass rate and a 13.9% mutation detection rate, representing a 17.6% absolute improvement over baseline and outperforming much larger general-purpose models. These results demonstrate that specialized training methodologies can significantly enhance LLM reasoning capabilities for hardware verification tasks, establishing a foundation for automated sub-unit testing in semiconductor design workflows.