mlsanityData to 7 Oct 2026

LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation

arXiv 2310.04535 ↗First posted 6 Oct 2023cs.LG cs.ARcited by 21

Not on the map. Why?

Abstract (arXiv)

Hardware design verification (DV) is a process that checks the functional equivalence of a hardware design against its specifications, improving hardware reliability and robustness. A key task in the DV process is the test stimuli generation, which creates a set of conditions or inputs for testing. These test conditions are often complex and specific to the given hardware design, requiring substantial human engineering effort to optimize. We seek a solution of automated and efficient testing for arbitrary hardware designs that takes advantage of large language models (LLMs). LLMs have already shown promising results for improving hardware design automation, but remain under-explored for hardware DV. In this paper, we propose an open-source benchmarking framework named LLM4DV that efficiently orchestrates LLMs for automated hardware test stimuli generation. Our analysis evaluates six different LLMs involving six prompting improvements over eight hardware designs and provides insight for future work on LLMs development for efficient automated DV.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 15 of its 36 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

15 of 36 references.

References · 15

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  7. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  8. 2022CodeT: Code Generation with Generated Testscited by 369
  9. 2023An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generationcited by 193
  10. 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 143
  11. 2023Chip-Chat: Challenges and Opportunities in Conversational Hardware Designcited by 136
  12. 2022Benchmarking Large Language Models for Automated Verilog RTL Code Generationcited by 112
3 more
  1. 2023GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Modelscited by 59
  2. 2022Pynguin: Automated Unit Test Generation for Pythoncited by 49
  3. 2023Unlocking Hardware Security Assurance: The Potential of LLMscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 9; 2026: 8.

21 papers.

Sort:

By area:

  1. 2024Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generationcited by 45
  2. 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verificationcited by 41
  3. 2025VFlow: Discovering Optimal Agentic Workflows for Verilog Generationcited by 19
  4. 2024Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Reviewcited by 17
  5. 2024VerilogReader: LLM-Aided Hardware Test Generationcited by 16
  6. 2024Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledgecited by 10
  7. 2025Enhancing Large Language Models for Hardware Verification: A Novel SystemVerilog Assertion Datasetcited by 10
  8. 2025VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generationcited by 8
  9. 2025SV-LLM: An Agentic Approach for SoC Security Verification using Large Language Modelscited by 5
  10. 2025Hey AI, Generate Me a Hardware Code! Agentic AI-based Hardware Design & Verificationcited by 5
  11. 2026GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generationcited by 4
  12. 2025HDLxGraph: Bridging Large Language Models and HDL Repositories via HDL Graph Databasescited by 1
9 more
  1. 2025Principle-Guided Verilog Optimization: IP-Safe Knowledge Transfer via Local-Cloud Collaborationcited by 1
  2. 2025DeepV: A Model-Agnostic Retrieval-Augmented Framework for Verilog Code Generation with a High-Quality Knowledge Basecited by 1
  3. 2026LAUDE: LLM-Assisted Unit Test Generation and Debugging of Hardware DEsignscited by 1
  4. 2026HAVEN: Hybrid Automated Verification ENgine for UVM Testbench Synthesis with LLMscited by 1
  5. 2026HierSVA: A Data Synthesis Pipeline, Dataset, and Benchmark for LLM-Driven Hierarchical Hardware Formal Verificationcited by 0
  6. 2026Agentic Electronic Design Automation: A Handoff Perspectivecited by 0
  7. 2026LLM for EDA in Front-End Design: Challenges and Opportunitiescited by 0
  8. 2026AgentDV: Closed-Loop Agentic AI for Hardware Design Verificationcited by 0
  9. 2026Trust, but Validate the Instrument: Auditing AI-Generated RTL Verification Plans on Authored Security-Regression Proxiescited by 0