NoTB: Oracle-Free Triage of LLM-Generated RTL via Cross-Model Formal Consensus
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used to generate register-transfer-level (RTL) designs from natural-language specifications. However, assessing functional correctness at early stages remains a fundamental challenge. Existing oracle-free approaches rely either on simulation-based agreement, which depends on LLM-generated testbenches that can fail or vary across models, or on LLM-as-a-judge heuristics, which produce inconsistent predictions. We introduce NoTB, an oracle-free triage framework that infers correctness from cross-model formal consensus. NoTB generates RTL implementations from multiple independently trained LLM families and applies Sequential Equivalence Checking (SEC) to identify designs that are provably equivalent. We show that the diversity of model families within an SEC-equivalent cluster induces a calibrated correctness signal, enabling risk-coverage tradeoffs without requiring testbenches. On 78 CVDP RTL-generation tasks, four-family formal consensus achieves 94.7% precision at 27% coverage; three-family consensus achieves 87% precision at 33% coverage. These operating points give designers a tunable accept/defer rule before a trusted testbench or golden RTL is available. Overall, NoTB demonstrates that formal cross-model agreement provides a reliable basis for high-confidence triage without model-dependent oracles
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 22 Aug 2026.
Builds on
15 of 21 references.
References · 15
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2022CodeT: Code Generation with Generated Testscited by 384
- 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
- 2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 148
- 2022Benchmarking Large Language Models for Automated Verilog RTL Code Generationcited by 115
- 2026Qwen3-Coder-Next Technical Reportcited by 71
- 2024Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generationcited by 45
- 2025Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verificationcited by 44
- 2024AssertLLM: Generating Hardware Verification Assertions from Design Specifications via Multi-LLMscited by 31
3 more
- 2024Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Modelscited by 30
- 2025Spec2RTL-Agent: Automated Hardware Code Generation from Complex Specifications Using LLM Agent Systemscited by 18
- 2025VRank: Enhancing Verilog Code Generation from Large Language Models via Self-Consistencycited by 11
Built on by
Citing papers per year: 2026: 1.
1 paper.