# HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.13536 · HTML: https://mlsanity.com/paper/2605.13536/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 13 May 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 17 of 17)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs](https://mlsanity.com/paper/2401.03868/) (2024)
- [C2HLSC: Leveraging Large Language Models to Bridge the Software-to-Hardware Design Gap](https://mlsanity.com/paper/2412.00214/) (2024)
- [C2HLSC: Can LLMs Bridge the Software-to-Hardware Design Gap?](https://mlsanity.com/paper/2406.09233/) (2024)
- [Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysis](https://mlsanity.com/paper/2502.13921/) (2025)
- [HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks](https://mlsanity.com/paper/2504.12268/) (2025)
- [SAGE-HLS: Syntax-Aware AST-Guided LLM for High-Level Synthesis Code Generation](https://mlsanity.com/paper/2508.03558/) (2025)
- [SynthAI: A Multi Agent Generative AI Framework for Automated Modular HLS Design Generation](https://mlsanity.com/paper/2405.16072/) (2024)
- [HLStrans: Dataset for C-to-HLS Hardware Code Synthesis](https://mlsanity.com/paper/2507.04315/) (2025)
- [LightRW: FPGA Accelerated Graph Dynamic Random Walks](https://mlsanity.com/paper/2304.07004/) (2023)
- [Data Processing with FPGAs on Modern Architectures](https://mlsanity.com/paper/2304.03044/) (2023)

## Built on by (most cited first, 1 of 1)

- [HLSmith: An Expert-Guided Agentic Framework for C/C++-to-HLS Translation](https://mlsanity.com/paper/2608.06791/) (2026)

## Abstract (arXiv)

High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR)---latency and resource utilization---critically governed by pragma configurations and code structure. Existing natural-language-to-HLS (NL-to-HLS) training approaches prioritize functional correctness while largely ignoring QoR. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results---only relative comparisons between candidates. Based on this insight, we propose HLS-Seek, a QoR-aware NL-to-HLS framework that avoids full synthesis-in-the-loop RL via a comparative proxy reward model achieving 99.53% Pareto-dominance accuracy. To prevent reward hacking, we introduce textituncertainty-aware Monte Carlo (MC) dropout switching that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 84.7% syntax correctness pass@1 and 81.4% functional correctness pass@5 on HLS-Eval~citeabikaram2025hlseval with only 7B parameters, surpassing GPT-5.1 on functional pass@5, while achieving 8.5× faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 19/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels.