Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Verification consumes the majority of modern chip design effort, yet the formal verification tools that provide mathematical guarantees of correctness remain expensive and restrictively licensed. While large language models (LLMs) have shown promise for hardware design, existing approaches to RTL repair validate their results through simulation - which exercises only a subset of inputs - or rely on commercial tools, and few combine formal proof with an entirely open-source toolchain. In this paper, we present a multi-agent pipeline that couples an LLM with an open-source formal backend (Yosys, SymbiYosys, and Z3) to repair RTL through counterexample-guided iteration: the framework generates formal properties, verifies the design, and feeds counterexamples back to the LLM until the design is proved correct by k-induction or an iteration budget is exhausted. Through an ALU case study, we show that the pipeline can detect and repair a real functional bug with a formal proof of correctness. Across a six-benchmark suite, one design is repaired reliably, and we characterize four distinct failure modes: bounded-cover vacuity, specification ambiguity, temporal-logic bugs, and multi-property pressure. We frame this work as a feasibility study with a detailed failure analysis, and additionally report a practical limitation of the Yosys bind directive relevant to the open-source formal verification community.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 4 of its 9 references that are in the map. First posted on arXiv 30 Jul 2026.
Builds on
4 of 9 references.
References · 4
- 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
- 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 200
- 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 148
- 2023RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Modelscited by 100
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification
- 2026GoGoTB: Agentic RTL Verification with Specification-Grounded Coverage Closure
- 2026Veri-Sure: A Contract-Aware Multi-Agent Framework with Temporal Tracing and Formal Verification for Correct RTL Code Generation
- 2026AgentDV: Closed-Loop Agentic AI for Hardware Design Verification
- 2026ACE-RTL: When Agentic Context Evolution Meets RTL-Specialized LLMs
- 2026Clover: A Neural-Symbolic Agentic Harness with Stochastic Tree-of-Thoughts for Verified RTL Repair
- 2026Interpretable and Verifiable Hardware Generation with LLM-Driven Stepwise Refinement
- 2026A Survey on LLM-Integrated Hardware Design Verification