mlsanityData to 14 Oct 2026

Reward-Weighted On-Policy Distillation with an Open Property-Equivalence Verifier for NL-to-SVA Generation

arXiv 2605.13501 ↗First posted 13 May 2026cs.AR cs.LGcited by 3

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

LLM-based generation of SystemVerilog Assertions (SVA) is often reported as nearing saturation, with the strongest specialized model reaching ~76% accuracy on NL2SVA-Human. We show that this aggregate hides a temporal gap: models that appear strong overall still collapse to a few implication templates on bounded-delay and liveness specifications. The core issue is that the dominant recipe, supervised fine-tuning on NL/SVA pairs, optimizes token-level mimicry rather than the property equivalence that defines SVA correctness. We introduce Reward-Weighted On-Policy Distillation (RWOPD), an on-policy distillation method that samples student rollouts, scores them with an open SymbiYosys+Z3 Property-Equivalence Checker (PEC), and applies a verifier-reward-weighted forward-KL gradient from a frozen 14B teacher on verifier-passable rollouts. This keeps the supervision dense at every response token while grounding both selection and loss weight in property-equivalent behavior. RWOPD distills CodeV-SVA-14B into a Qwen2.5-Coder-7B-Instruct student that sets a new state of the art on NL2SVA-Human and NL2SVA-Machine across pass@1, pass@5, and pass@10, surpassing both specialized prior SOTA models and 671B general-purpose baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 14 of its 22 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

14 of 22 references.

References · 14

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  8. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  9. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  10. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  11. 2024AssertLLM: Generating Hardware Verification Assertions from Design Specifications via Multi-LLMscited by 31
  12. 2024FVEval: Understanding Language Model Capabilities in Formal Verification of Digital Hardwarecited by 10
2 more
  1. 2025AssertFix: Empowering Automated Assertion Fix via Large Language Modelscited by 3
  2. 2026QiMeng-CodeV-SVA: Training Specialized LLMs for Hardware Assertion Generation via RTL-Grounded Bidirectional Data Synthesiscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reward-Gated On-Policy Distillation
  2. 2026KL for a KL: On-Policy Distillation with Control Variate Baseline
  3. 2026Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
  4. 2026Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
  5. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
  6. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
  7. 2026On the Geometry of On-Policy Distillation
  8. 2026PowerOPD: Stabilizing On-Policy Distillation with Bounded Power Transformation