Reward-Weighted On-Policy Distillation with an Open Property-Equivalence Verifier for NL-to-SVA Generation
On the map: Knowledge distillation → On-policy and self-distillation for LLMs
Abstract (arXiv)
LLM-based generation of SystemVerilog Assertions (SVA) is often reported as nearing saturation, with the strongest specialized model reaching ~76% accuracy on NL2SVA-Human. We show that this aggregate hides a temporal gap: models that appear strong overall still collapse to a few implication templates on bounded-delay and liveness specifications. The core issue is that the dominant recipe, supervised fine-tuning on NL/SVA pairs, optimizes token-level mimicry rather than the property equivalence that defines SVA correctness. We introduce Reward-Weighted On-Policy Distillation (RWOPD), an on-policy distillation method that samples student rollouts, scores them with an open SymbiYosys+Z3 Property-Equivalence Checker (PEC), and applies a verifier-reward-weighted forward-KL gradient from a frozen 14B teacher on verifier-passable rollouts. This keeps the supervision dense at every response token while grounding both selection and loss weight in property-equivalent behavior. RWOPD distills CodeV-SVA-14B into a Qwen2.5-Coder-7B-Instruct student that sets a new state of the art on NL2SVA-Human and NL2SVA-Machine across pass@1, pass@5, and pass@10, surpassing both specialized prior SOTA models and 671B general-purpose baselines.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 14 of its 22 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
14 of 22 references.
References · 14
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
- 2024AssertLLM: Generating Hardware Verification Assertions from Design Specifications via Multi-LLMscited by 31
- 2024FVEval: Understanding Language Model Capabilities in Formal Verification of Digital Hardwarecited by 10
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026A Formula-Driven Survey and Research Agenda for On-Policy Distillationcited by 6
- 2026Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoningcited by 4
- 2026Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillationcited by 3
- 2026Beyond Token-Local Imitation: Reward-Compatible Temporal Credit Assignment for On-Policy Distillation15 Sep 2026
- 2026Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning19 Aug 2026
- 2026A Formula-Driven Survey and Research Agenda for On-Policy Distillation22 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026Reward-Gated On-Policy Distillation
- 2026KL for a KL: On-Policy Distillation with Control Variate Baseline
- 2026Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
- 2026Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
- 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
- 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
- 2026On the Geometry of On-Policy Distillation
- 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation