# Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant

arXiv: https://arxiv.org/abs/2508.20907 · HTML: https://mlsanity.com/paper/2508.20907/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 27 of 27)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https://mlsanity.com/paper/2305.07922/) (2023)
- [Arcee's MergeKit: A Toolkit for Merging Large Language Models](https://mlsanity.com/paper/2403.13257/) (2024)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models](https://mlsanity.com/paper/2406.13542/) (2024)
- [Training a Scientific Reasoning Model for Chemistry](https://mlsanity.com/paper/2506.17238/) (2025)
- [Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models](https://mlsanity.com/paper/2406.14712/) (2024)
- [Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code](https://mlsanity.com/paper/2405.19495/) (2024)
- [Agent-Q: Fine-Tuning Large Language Models for Quantum Circuit Generation and Optimization](https://mlsanity.com/paper/2504.11109/) (2025)
- [Enhancing LLM-based Quantum Code Generation with Multi-Agent Optimization and Quantum Error Correction](https://mlsanity.com/paper/2504.14557/) (2025)
- [QHackBench: Benchmarking Large Language Models for Quantum Code Generation Using PennyLane Hackathon Challenges](https://mlsanity.com/paper/2506.20008/) (2025)
- [PennyCoder: Efficient Domain-Specific LLMs for PennyLane-Based Quantum Code Generation](https://mlsanity.com/paper/2507.19562/) (2025)
- [Q-Fusion: Diffusing Quantum Circuits](https://mlsanity.com/paper/2504.20794/) (2025)
- [Taxonomy of migration scenarios for Qiskit refactoring using LLMs](https://mlsanity.com/paper/2506.07135/) (2025)
- [QSpark: Towards Reliable Qiskit Code Generation](https://mlsanity.com/paper/2507.12642/) (2025)
- [Automatic Qiskit Code Refactoring Using Large Language Models](https://mlsanity.com/paper/2506.14535/) (2025)

## Built on by (most cited first, 6 of 6)

- [QUASAR: Quantum Assembly Code Generation Using Tool-Augmented LLMs via Agentic RL](https://mlsanity.com/paper/2510.00967/) (2025)
- [QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation](https://mlsanity.com/paper/2604.08570/) (2026)
- [Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation](https://mlsanity.com/paper/2605.27210/) (2026)
- [Aligning Quantum Operators with Large Language Models](https://mlsanity.com/paper/2606.13811/) (2026)
- [Generative AI for Quantum Circuits and Quantum Code: A Technical Review and Taxonomy](https://mlsanity.com/paper/2603.16216/) (2026)
- [Fine-Tuning Large Language Models for Quantum Reasoning](https://mlsanity.com/paper/2606.21974/) (2026)

## Abstract (arXiv)

Qiskit is an open-source quantum computing framework that allows users to design, simulate, and run quantum circuits on real quantum hardware. We explore post-training techniques for LLMs to assist in writing Qiskit code. We introduce quantum verification as an effective method for ensuring code quality and executability on quantum hardware. To support this, we developed a synthetic data pipeline that generates quantum problem-unit test pairs and used it to create preference data for aligning LLMs with DPO. Additionally, we trained models using GRPO, leveraging quantum-verifiable rewards provided by the quantum hardware. Our best-performing model, combining DPO and GRPO, surpasses the strongest open-source baselines on the challenging Qiskit-HumanEval-hard benchmark.