# LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation

arXiv: https://arxiv.org/abs/2310.04535 · HTML: https://mlsanity.com/paper/2310.04535/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model](https://mlsanity.com/paper/2308.05345/) (2023)
- [Chip-Chat: Challenges and Opportunities in Conversational Hardware Design](https://mlsanity.com/paper/2305.13243/) (2023)
- [Benchmarking Large Language Models for Automated Verilog RTL Code Generation](https://mlsanity.com/paper/2212.11140/) (2022)
- [GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models](https://mlsanity.com/paper/2309.10730/) (2023)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [Unlocking Hardware Security Assurance: The Potential of LLMs](https://mlsanity.com/paper/2308.11042/) (2023)

## Built on by (most cited first, 23 of 23)

- [Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generation](https://mlsanity.com/paper/2408.11053/) (2024)
- [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://mlsanity.com/paper/2506.14074/) (2025)
- [VFlow: Discovering Optimal Agentic Workflows for Verilog Generation](https://mlsanity.com/paper/2504.03723/) (2025)
- [Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review](https://mlsanity.com/paper/2401.01519/) (2024)
- [VerilogReader: LLM-Aided Hardware Test Generation](https://mlsanity.com/paper/2406.04373/) (2024)
- [Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge](https://mlsanity.com/paper/2402.01728/) (2024)
- [Enhancing Large Language Models for Hardware Verification: A Novel SystemVerilog Assertion Dataset](https://mlsanity.com/paper/2503.08923/) (2025)
- [VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generation](https://mlsanity.com/paper/2507.13369/) (2025)
- [SV-LLM: An Agentic Approach for SoC Security Verification using Large Language Models](https://mlsanity.com/paper/2506.20415/) (2025)
- [Hey AI, Generate Me a Hardware Code! Agentic AI-based Hardware Design & Verification](https://mlsanity.com/paper/2507.02660/) (2025)
- [GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation](https://mlsanity.com/paper/2601.07593/) (2026)
- [HAVEN: Hybrid Automated Verification ENgine for UVM Testbench Synthesis with LLMs](https://mlsanity.com/paper/2604.27643/) (2026)
- [HDLxGraph: Bridging Large Language Models and HDL Repositories via HDL Graph Databases](https://mlsanity.com/paper/2505.15701/) (2025)
- [Principle-Guided Verilog Optimization: IP-Safe Knowledge Transfer via Local-Cloud Collaboration](https://mlsanity.com/paper/2508.05675/) (2025)
- [DeepV: A Model-Agnostic Retrieval-Augmented Framework for Verilog Code Generation with a High-Quality Knowledge Base](https://mlsanity.com/paper/2510.05327/) (2025)
- [LAUDE: LLM-Assisted Unit Test Generation and Debugging of Hardware DEsigns](https://mlsanity.com/paper/2601.08856/) (2026)
- [HierSVA: A Data Synthesis Pipeline, Dataset, and Benchmark for LLM-Driven Hierarchical Hardware Formal Verification](https://mlsanity.com/paper/2606.13706/) (2026)
- [Agentic Electronic Design Automation: A Handoff Perspective](https://mlsanity.com/paper/2606.19795/) (2026)
- [LLM for EDA in Front-End Design: Challenges and Opportunities](https://mlsanity.com/paper/2607.09616/) (2026)
- [AgentDV: Closed-Loop Agentic AI for Hardware Design Verification](https://mlsanity.com/paper/2608.27148/) (2026)
- [Trust, but Validate the Instrument: Auditing AI-Generated RTL Verification Plans on Authored Security-Regression Proxies](https://mlsanity.com/paper/2609.19844/) (2026)
- [A Survey on LLM-Integrated Hardware Design Verification](https://mlsanity.com/paper/2610.10580/) (2026)
- [Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure](https://mlsanity.com/paper/2610.17449/) (2026)

## Abstract (arXiv)

Hardware design verification (DV) is a process that checks the functional equivalence of a hardware design against its specifications, improving hardware reliability and robustness. A key task in the DV process is the test stimuli generation, which creates a set of conditions or inputs for testing. These test conditions are often complex and specific to the given hardware design, requiring substantial human engineering effort to optimize. We seek a solution of automated and efficient testing for arbitrary hardware designs that takes advantage of large language models (LLMs). LLMs have already shown promising results for improving hardware design automation, but remain under-explored for hardware DV. In this paper, we propose an open-source benchmarking framework named LLM4DV that efficiently orchestrates LLMs for automated hardware test stimuli generation. Our analysis evaluates six different LLMs involving six prompting improvements over eight hardware designs and provides insight for future work on LLMs development for efficient automated DV.