Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation
Not on the map. Why?
Abstract (arXiv)
In last two years, large language models (LLMs) have shown strong capabilities in code generation, including hardware design at register-transfer level (RTL). While their use in high-level synthesis (HLS) remains comparatively less mature, the ratio of HLS- to RTL-focused studies has shifted from 1:10 to 2:10 in the past six months, indicating growing interest in leveraging LLMs for high-level design entry while relying on downstream synthesis for optimization. This growing trend highlights the need for a comprehensive benchmarking and evaluation framework dedicated to LLM-based HLS. To address this, We present Bench4HLS for evaluating LLM-generated HLS designs. Bench4HLS comprises 170 manually drafted and validated case studies, spanning small kernels to complex accelerators, curated from widely used public repositories. The framework supports fully automated assessment of compilation success, functional correctness via simulation, and synthesis feasibility/optimization. Crucially, Bench4HLS integrates a pluggable API for power, performance, and area (PPA) analysis across various HLS toolchains and architectures, demonstrated here with Xilinx Vitis HLS and validated on Catapult HLS. By providing a structured, extensible, and plug-and-play testbed, Bench4HLS establishes a foundational methodology for benchmarking LLMs in HLS workflows.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 16 Jan 2026.
Builds on
24 of 41 references.
References · 24
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
- 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 143
- 2021hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devicescited by 61
- 2024Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generationcited by 45
- 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verificationcited by 41
- 2025A Survey of Research in Large Language Models for Electronic Design Automationcited by 38
- 2024HLSPilot: LLM-based High-Level Synthesiscited by 24
- 2024Automated C/C++ Program Repair for High-Level Synthesis via Large Language Modelscited by 18
- 2025HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDLcited by 17
- 2025RTL++: Graph-enhanced LLM for RTL Code Generationcited by 16
- 2024C2HLSC: Can LLMs Bridge the Software-to-Hardware Design Gap?cited by 12
12 more
- 2025Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysiscited by 10
- 2025HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Taskscited by 10
- 2018Parallel Programming for FPGAscited by 9
- 2025Can Reasoning Models Reason about Hardware? An Agentic HLS Perspectivecited by 9
- 2025SAGE-HLS: Syntax-Aware AST-Guided LLM for High-Level Synthesis Code Generationcited by 9
- 2025LIFT: LLM-Based Pragma Insertion for HLS via GNN Supervised Fine-Tuningcited by 8
- 2024Are LLMs Any Good for High-Level Synthesis?cited by 6
- 2024Optimizing High-Level Synthesis Designs with Retrieval-Augmented Large Language Modelscited by 6
- 2024SynthAI: A Multi Agent Generative AI Framework for Automated Modular HLS Design Generationcited by 5
- 2025TimelyHLS: LLM-Based Timing-Aware and Architecture-Specific FPGA HLS Optimizationcited by 5
- 2024Agentic-HLS: An agentic reasoning based high-level synthesis system using large language models (AI for EDA workshop 2024)cited by 4
- 2025DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.