mlsanityData to 13 Oct 2026

Evaluating LLMs for Hardware Design and Test

arXiv 2405.02326 ↗First posted 23 Apr 2024cs.AR cs.AI cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated capabilities for producing code in Hardware Description Languages (HDLs). However, most of the focus remains on their abilities to write functional code, not test code. The hardware design process consists of both design and test, and so eschewing validation and verification leaves considerable potential benefit unexplored, given that a design and test framework may allow for progress towards full automation of the digital design pipeline. In this work, we perform one of the first studies exploring how a LLM can both design and test hardware modules from provided specifications. Using a suite of 8 representative benchmarks, we examined the capabilities and limitations of the state-of-the-art conversational LLMs when producing Verilog for functional and verification purposes. We taped out the benchmarks on a Skywater 130nm shuttle and received the functional chip.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 11 of its 26 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

11 of 26 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,106
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,205
  4. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
  5. 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 200
  6. 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 174
  7. 2023Chip-Chat: Challenges and Opportunities in Conversational Hardware Designcited by 140
  8. 2022Benchmarking Large Language Models for Automated Verilog RTL Code Generationcited by 115
  9. 2020DAVE: Deriving Automatically Verilog from Englishcited by 48
  10. 2023(Security) Assertions by Large Language Modelscited by 32
  11. 2023FLAG: Finding Line Anomalies (in code) with Generative AIcited by 7

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 2.

8 papers.

  1. 2024Automatically Improving LLM-based Verilog Generation using EDA Tool Feedbackcited by 19
  2. 2025Can Reasoning Models Reason about Hardware? An Agentic HLS Perspectivecited by 9
  3. 2025Large Language Model for Verilog Code Generation: Literature Review and the Road Aheadcited by 8
  4. 2025Large Language Models (LLMs) for Electronic Design Automation (EDA)cited by 3
  5. 2025Deep Representation Learning for Electronic Design Automationcited by 0
  6. 2025Controlling Context: Generative AI at Work in Integrated Circuit Design and Other High-Precision Domainscited by 0
  7. 2026OpenRTLSet: A Fully Open-Source Dataset for Large Language Model-based Verilog Module Designcited by 0
  8. 2026Agentic Electronic Design Automation: A Handoff Perspectivecited by 0