QuanBench: Benchmarking Quantum Code Generation with Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have demonstrated good performance in general code generation; however, their capabilities in quantum code generation remain insufficiently studied. This paper presents QuanBench, a benchmark for evaluating LLMs on quantum code generation. QuanBench includes 44 programming tasks that cover quantum algorithms, state preparation, gate decomposition, and quantum machine learning. Each task has an executable canonical solution and is evaluated by functional correctness (Pass@K) and quantum semantic equivalence (Process Fidelity). We evaluate several recent LLMs, including general-purpose and code-specialized models. The results show that current LLMs have limited capability in generating the correct quantum code, with overall accuracy below 40% and frequent semantic errors. We also analyze common failure cases, such as outdated API usage, circuit construction errors, and incorrect algorithm logic. QuanBench provides a basis for future work on improving quantum code generation with LLMs.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Oct 2025.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computingcited by 5
- 2026QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generationcited by 2
- 2026Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluationcited by 2
- 2026QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debuggingcited by 2
- 2026QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learningcited by 1
- 2026PennySynth: RAG-Driven Data Synthesis for Automated Quantum Code Generationcited by 0
- 2026QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuitscited by 0
- 2026Leveraging LLM-Based Agentic Systems to Generate Quantum Applications for Test Optimizationcited by 0
- 2026LLM-Driven Algorithm Design for Quantum Circuit Synthesis based on Binary Decision Diagramscited by 0
- 2026AG-CoT: Verified Algorithmic Traces for LLM Program Synthesis on Clifford Circuitscited by 0
- 2026AG-CoT: Verified Algorithmic Traces for LLM Program Synthesis on Clifford Circuits27 Sep 2026
- 2026LLM-Driven Algorithm Design for Quantum Circuit Synthesis based on Binary Decision Diagrams4 Sep 2026
- 2026Leveraging LLM-Based Agentic Systems to Generate Quantum Applications for Test Optimization1 Jul 2026
- 2026QBugLM: An Agentic Benchmarking Framework for LLM-based Quantum Software Debugging5 Jun 2026
- 2026Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation26 May 2026
- 2026PennySynth: RAG-Driven Data Synthesis for Automated Quantum Code Generation25 May 2026
- 2026QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuits28 Apr 2026
- 2026QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning20 Apr 2026
- 2026QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation25 Mar 2026
- 2026Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing10 Feb 2026