FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?
Not on the map. Why?
Abstract (arXiv)
Can large language models generate not just correct, but fast hardware? This paper investigates the question in financial FPGA design, where 5-10 nanoseconds of latency determines competitive advantage and designs iterate continuously as protocols, strategies, and regulations evolve. FinHardBench, a benchmark of 33 financial computing tasks, is presented together with three experiments that mirror the real-world FPGA iteration cycle: generating new modules from specifications, tuning system-level configurations across a 6-stage trading pipeline, and adapting existing modules to specification changes. Evaluation of six LLMs on 1530+ experiment rounds yields three findings: (1) models achieve 19-61% functional correctness with timing degradation up to 13.7× on specific tasks; (2) in system-level design space exploration, top LLMs converge to the optimal configuration with higher reliability than random search, simulated annealing, and Bayesian optimization baselines (5/5 seeds vs. 0-4/5 at the same 24-round budget); (3) strategy-level specification changes remain unsolved for most models. Across the six models, generation and DSE rankings overlap moderately: the strongest code generator is not the fastest architecture optimizer, and the weakest code generator (MiniMax M2.7) still reaches the system optimum on 4 of 5 seeds. On the tasks in FinHardBench, difficulty tracks training data pattern availability more closely than abstraction level. FinHardBench is released as an open-source benchmark.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 22 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
10 of 22 references.
References · 10
- 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
- 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 197
- 2023ChipNeMo: Domain-Adapted LLMs for Chip Designcited by 168
- 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 143
- 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verificationcited by 41
- 2024CodeV: Empowering LLMs with HDL Generation through Multi-Level Summarizationcited by 38
- 2025AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decaycited by 36
- 2024Mercury: A Code Efficiency Benchmark for Code Large Language Modelscited by 34
- 2024AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factorscited by 25
- 2025LLM-VeriPPA: Power, Performance, and Area Optimization aware Verilog Code Generation with Large Language Modelscited by 5
Built on by
Not cited yet.