# VHDLSuite: Unified Pipeline for LLM VHDL Generation with Data Synthesis and Evaluation

arXiv: https://arxiv.org/abs/2606.13735 · HTML: https://mlsanity.com/paper/2606.13735/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 14 of 14)

- [VerilogEval: Evaluating Large Language Models for Verilog Code Generation](https://mlsanity.com/paper/2309.07544/) (2023)
- [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [AutoChip: Automating HDL Generation Using LLM Feedback](https://mlsanity.com/paper/2311.04887/) (2023)
- [Towards LLM-Powered Verilog RTL Assistant: Self-Verification and Self-Correction](https://mlsanity.com/paper/2406.00115/) (2024)
- [Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges](https://mlsanity.com/paper/2412.03220/) (2024)
- [LLMs and the Future of Chip Design: Unveiling Security Risks and Building Trust](https://mlsanity.com/paper/2405.07061/) (2024)
- [Automatically Improving LLM-based Verilog Generation using EDA Tool Feedback](https://mlsanity.com/paper/2411.11856/) (2024)
- [VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation](https://mlsanity.com/paper/2406.04379/) (2024)
- [AIvril: AI-Driven RTL Generation With Verification In-The-Loop](https://mlsanity.com/paper/2409.11411/) (2024)
- [NetDeTox: Adversarial and Efficient Evasion of Hardware-Security GNNs via RL-LLM Orchestration](https://mlsanity.com/paper/2512.00119/) (2025)
- [Chain-of-Descriptions: Improving Code LLMs for VHDL Code Generation and Summarization](https://mlsanity.com/paper/2507.12308/) (2025)
- [VeriDispatcher: Multi-Model Dispatching through Pre-Inference Difficulty Prediction for RTL Generation Optimization](https://mlsanity.com/paper/2511.22749/) (2025)
- [Evolutionary Large Language Models for Hardware Security: A Comparative Survey](https://mlsanity.com/paper/2404.16651/) (2024)

## Abstract (arXiv)

Large Language Models (LLM) have shown impressive capabilities in Register Transfer Level (RTL) code generation, particularly for Verilog. However, evaluating their performance with other Hardware Description Languages (HDL), especially VHDL, remains limited although its distinct language characteristics, such as stricter semantic rules, introduce evaluation considerations that differ from Verilog. This lack of coverage restricts fully understanding of how well current models generalize across hardware design languages with differing structures and semantics. To address this gap, we introduce VHDLSuite, a benchmark-centered infrastructure for scalable VHDL generation evaluation, integrating automated benchmark synthesis, executable validation, and multi-model diagnostic analysis. First, we propose a data pipeline that automatically converts Verilog designs and their accompanying testbenches into executable VHDL benchmark instances, followed by VUnit/GHDL-based validation to ensure each released task is compilable, runnable, and consistently checkable in the VHDL environment. Second, we introduce VHDLBench, a benchmark with over 200 VHDL problems with complete and validated testbenches across a wide range of complexity levels. Third, we extensively evaluate cutting-edge LLMs and uncover key challenges specific on LLM-aided VHDL generation. Our findings provide important insights and support future work in multi-language hardware design automation.Our data pipeline, benchmark, and evaluation framework will be open-sourced.