# TB or Not TB: Coverage-Driven Direct Preference Optimization for Verilog Stimulus Generation

arXiv: https://arxiv.org/abs/2511.15767 · HTML: https://mlsanity.com/paper/2511.15767/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Process reward models for reasoning](https://mlsanity.com/alignment/process-reward-models-for-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [β-DPO: Direct Preference Optimization with Dynamic β](https://mlsanity.com/paper/2407.08639/) (2024)
- [VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool](https://mlsanity.com/paper/2408.08927/) (2024)
- [WPO: Enhancing RLHF with Weighted Preference Optimization](https://mlsanity.com/paper/2406.11827/) (2024)
- [Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification](https://mlsanity.com/paper/2506.14074/) (2025)
- [AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design](https://mlsanity.com/paper/2407.03891/) (2024)
- [AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization](https://mlsanity.com/paper/2410.10148/) (2024)
- [CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design](https://mlsanity.com/paper/2411.08510/) (2024)
- [RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization](https://mlsanity.com/paper/2410.04203/) (2024)
- [PyraNet: A Multi-Layered Hierarchical Dataset for Verilog](https://mlsanity.com/paper/2412.06947/) (2024)
- [VerilogReader: LLM-Aided Hardware Test Generation](https://mlsanity.com/paper/2406.04373/) (2024)
- [A Multi-Expert Large Language Model Architecture for Verilog Code Generation](https://mlsanity.com/paper/2404.08029/) (2024)
- [VeriThoughts: Enabling Automated Verilog Code Generation using Reasoning and Formal Verification](https://mlsanity.com/paper/2505.20302/) (2025)
- [AssertionForge: Enhancing Formal Verification Assertion Generation with Structured Representation of Specifications and RTL](https://mlsanity.com/paper/2503.19174/) (2025)
- [VeriMind: Agentic LLM for Automated Verilog Generation with a Novel Evaluation Metric](https://mlsanity.com/paper/2503.16514/) (2025)
- [Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts](https://mlsanity.com/paper/2412.04628/) (2024)
- [KL Penalty Control via Perturbation for Direct Preference Optimization](https://mlsanity.com/paper/2502.13177/) (2025)

## Built on by (most cited first, 4 of 4)

- [LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation](https://mlsanity.com/paper/2602.16953/) (2026)
- [HAVEN: Hybrid Automated Verification ENgine for UVM Testbench Synthesis with LLMs](https://mlsanity.com/paper/2604.27643/) (2026)
- [CovR: Coverage-Aware Hardware Verification via Reasoning-Guided Reinforcement Learning](https://mlsanity.com/paper/2609.19189/) (2026)
- [A Survey on LLM-Integrated Hardware Design Verification](https://mlsanity.com/paper/2610.10580/) (2026)

## Abstract (arXiv)

With the rapid advancement of Large Language Models (LLMs), there is growing interest in applying them to hardware design and verification. Among these stages, design verification remains the most time-consuming and resource-intensive phase, where generating effective stimuli for the design under test (DUT) is both critical and labor-intensive. We present it TB or not TB, a framework for automated stimulus generation using LLMs fine-tuned through Coverage-Driven Direct Preference Optimization (CD-DPO). To enable preference-based training, we introduce PairaNet, a dataset derived from PyraNet that pairs high- and low-quality testbenches labeled using simulation-derived coverage metrics. The proposed CD-DPO method integrates quantitative coverage feedback directly into the optimization objective, guiding the model toward generating stimuli that maximize verification coverage. Experiments on the CVDP CID12 benchmark show that it TB or not TB outperforms both open-source and commercial baselines, achieving up to 77.27% improvement in code coverage, demonstrating the effectiveness of Coverage-driven preference optimization for LLM-based hardware verification.