mlsanityData to 7 Oct 2026

Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis

arXiv 2502.13921 ↗First posted 19 Feb 2025cs.LG cs.AR cs.SEcited by 10

Not on the map. Why?

Abstract (arXiv)

Recent advances in code generation have illuminated the potential of employing large language models (LLMs) for general-purpose programming languages such as Python and C++, opening new opportunities for automating software development and enhancing programmer productivity. The potential of LLMs in software programming has sparked significant interest in exploring automated hardware generation and automation. Although preliminary endeavors have been made to adopt LLMs in generating hardware description languages (HDLs), several challenges persist in this direction. First, the volume of available HDL training data is substantially smaller compared to that for software programming languages. Second, the pre-trained LLMs, mainly tailored for software code, tend to produce HDL designs that are more error-prone. Third, the generation of HDL requires a significantly higher number of tokens compared to software programming, leading to inefficiencies in cost and energy consumption. To tackle these challenges, this paper explores leveraging LLMs to generate High-Level Synthesis (HLS)-based hardware design. Although code generation for domain-specific programming languages is not new in the literature, we aim to provide experimental results, insights, benchmarks, and evaluation infrastructure to investigate the suitability of HLS over low-level HDLs for LLM-assisted hardware design generation. To achieve this, we first finetune pre-trained models for HLS-based hardware generation, using a collected dataset with text prompts and corresponding reference HLS designs. An LLM-assisted framework is then proposed to automate end-to-end hardware code generation, which also investigates the impact of chain-of-thought and feedback loops promoting techniques on HLS-design generation. Limited by the timeframe of this research, we plan to evaluate more advanced reasoning models in the future.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

17 of 31 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
  12. 2023Structured Chain-of-Thought Prompting for Code Generationcited by 136
5 more
  1. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  2. 2018Mapping Language to Code in Programmatic Contextcited by 104
  3. 2016Latent Predictor Networks for Code Generationcited by 73
  4. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 46
  5. 2015On the "Naturalness" of Buggy Codecited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Taskscited by 10
  2. 2025Enhancing LLM-based Quantum Code Generation with Multi-Agent Optimization and Quantum Error Correctioncited by 10
  3. 2026ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Designcited by 7
  4. 2025FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoningcited by 3
  5. 2026Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generationcited by 2
  6. 2026HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learningcited by 1
  7. 2025Advancing AI-assisted Hardware Design with Hierarchical Decentralized Training and Personalized Inference-Time Optimizationcited by 0
  8. 2026BigPower: Hierarchical Source-Level Module Power Estimation for CPUs with Large Language Modelscited by 0
  9. 2026LLM for EDA in Front-End Design: Challenges and Opportunitiescited by 0
  10. 2026HLSmith: An Expert-Guided Agentic Framework for C/C++-to-HLS Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead
  2. 2025HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL
  3. 2025SAGE-HLS: Syntax-Aware AST-Guided LLM for High-Level Synthesis Code Generation
  4. 2026LLM4RTL: Tool-Assisted LLM for RTL Generation
  5. 2024AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs
  6. 2025Holistic Evaluation of State-of-the-Art LLMs for Code Generation
  7. 2024Optimizing High-Level Synthesis Designs with Retrieval-Augmented Large Language Models
  8. 2026Exploring the Role of LLMs in HPC Programming: A Survey