# AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design

arXiv: https://arxiv.org/abs/2407.03891 · HTML: https://mlsanity.com/paper/2407.03891/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2024.

## Built on by (most cited first, 30 of 41)

- [CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair](https://mlsanity.com/paper/2409.12993/) (2024)
- [RTL-Breaker: Assessing the Security of LLMs against Backdoor Attacks on HDL Code Generation](https://mlsanity.com/paper/2411.17569/) (2024)
- [VeriContaminated: Assessing LLM-Driven Verilog Coding for Data Contamination](https://mlsanity.com/paper/2503.13572/) (2025)
- [SALAD: Systematic Assessment of Machine Unlearning on LLM-Aided Hardware Design](https://mlsanity.com/paper/2506.02089/) (2025)
- [Schemato -- An LLM for Netlist-to-Schematic Conversion](https://mlsanity.com/paper/2411.13899/) (2024)
- [VeriLeaky: Navigating IP Protection vs Utility in Fine-Tuning for LLM-Driven Verilog Coding](https://mlsanity.com/paper/2503.13116/) (2025)
- [CoopetitiveV: Leveraging LLM-powered Coopetitive Multi-Agent Prompting for High-quality Verilog Generation](https://mlsanity.com/paper/2412.11014/) (2024)
- [Revolution or Hype? Seeking the Limits of Large Models in Hardware Design](https://mlsanity.com/paper/2509.04905/) (2025)
- [NetDeTox: Adversarial and Efficient Evasion of Hardware-Security GNNs via RL-LLM Orchestration](https://mlsanity.com/paper/2512.00119/) (2025)
- [VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generation](https://mlsanity.com/paper/2507.13369/) (2025)
- [ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design](https://mlsanity.com/paper/2601.21448/) (2026)
- [Are LLMs Any Good for High-Level Synthesis?](https://mlsanity.com/paper/2408.10428/) (2024)
- [MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs](https://mlsanity.com/paper/2507.19525/) (2025)
- [Benchmark Everything Everywhere All at Once](https://mlsanity.com/paper/2606.06462/) (2026)
- [ResBench: Benchmarking LLM-Generated FPGA Designs with Resource Awareness](https://mlsanity.com/paper/2503.08823/) (2025)
- [HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation](https://mlsanity.com/paper/2601.13864/) (2026)
- [Towards LLM-based Root Cause Analysis of Hardware Design Failures](https://mlsanity.com/paper/2507.06512/) (2025)
- [TB or Not TB: Coverage-Driven Direct Preference Optimization for Verilog Stimulus Generation](https://mlsanity.com/paper/2511.15767/) (2025)
- [GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation](https://mlsanity.com/paper/2601.07593/) (2026)
- [Large Language Models (LLMs) for Electronic Design Automation (EDA)](https://mlsanity.com/paper/2508.20030/) (2025)
- [Pluto: A Benchmark for Evaluating Efficiency of LLM-generated Hardware Code](https://mlsanity.com/paper/2510.14756/) (2025)
- [CorrectHDL: Agentic HDL Design with LLMs Leveraging High-Level Synthesis as Reference](https://mlsanity.com/paper/2511.16395/) (2025)
- [QiMeng-CodeV-SVA: Training Specialized LLMs for Hardware Assertion Generation via RTL-Grounded Bidirectional Data Synthesis](https://mlsanity.com/paper/2603.14239/) (2026)
- [HAVEN: Hybrid Automated Verification ENgine for UVM Testbench Synthesis with LLMs](https://mlsanity.com/paper/2604.27643/) (2026)
- [Review of Machine Learning for Micro-Electronic Design Verification](https://mlsanity.com/paper/2503.11687/) (2025)
- [DeepV: A Model-Agnostic Retrieval-Augmented Framework for Verilog Code Generation with a High-Quality Knowledge Base](https://mlsanity.com/paper/2510.05327/) (2025)
- [GoGoTB: Agentic RTL Verification with Specification-Grounded Coverage Closure](https://mlsanity.com/paper/2607.26181/) (2026)
- [LLM-based Hardware Development with Hierarchical IRs and End-to-End Multi-Agent Workflow](https://mlsanity.com/paper/2608.30659/) (2026)
- [Lorecast: Layout-Aware Performance and Power Forecasting from Natural Language](https://mlsanity.com/paper/2503.11662/) (2025)
- [DUET: Agentic Design Understanding via Experimentation and Testing](https://mlsanity.com/paper/2512.06247/) (2025)

## Abstract (arXiv)

In digital circuit design, testbenches constitute the cornerstone of simulation-based hardware verification. Traditional methodologies for testbench generation during simulation-based hardware verification still remain partially manual, resulting in inefficiencies in testing various scenarios and requiring expensive time from designers. Large Language Models (LLMs) have demonstrated their potential in automating the circuit design flow. However, directly applying LLMs to generate testbenches suffers from a low pass rate. To address this challenge, we introduce AutoBench, the first LLM-based testbench generator for digital circuit design, which requires only the description of the design under test (DUT) to automatically generate comprehensive testbenches. In AutoBench, a hybrid testbench structure and a self-checking system are realized using LLMs. To validate the generated testbenches, we also introduce an automated testbench evaluation framework to evaluate the quality of generated testbenches from multiple perspectives. Experimental results demonstrate that AutoBench achieves a 57% improvement in the testbench pass@1 ratio compared with the baseline that directly generates testbenches using LLMs. For 75 sequential circuits, AutoBench successfully has a 3.36 times testbench pass@1 ratio compared with the baseline. The source codes and experimental results are open-sourced at this link: https://github.com/AutoBench/AutoBench