# QCircuitBench: A Large-Scale Dataset for Benchmarking Quantum Algorithm Design

arXiv: https://arxiv.org/abs/2410.07961 · HTML: https://mlsanity.com/paper/2410.07961/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 65 references that are in the map.
- First posted on arXiv 10 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [LeanDojo: Theorem Proving with Retrieval-Augmented Language Models](https://mlsanity.com/paper/2306.15626/) (2023)
- [LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset](https://mlsanity.com/paper/2402.09391/) (2024)
- [Quantum machine learning: a classical perspective](https://mlsanity.com/paper/1707.08561/) (2017)
- [Robust data encodings for quantum classifiers](https://mlsanity.com/paper/2003.01695/) (2020)
- [Quantum circuit synthesis with diffusion models](https://mlsanity.com/paper/2311.02041/) (2023)
- [Entangled Datasets for Quantum Machine Learning](https://mlsanity.com/paper/2109.03400/) (2021)
- [MNISQ: A Large-Scale Quantum Circuit Dataset for Machine Learning on/for Quantum Computers in the NISQ era](https://mlsanity.com/paper/2306.16627/) (2023)

## Built on by (most cited first, 15 of 15)

- [Structured Scaling of AI Discovery Across Diverse Scientific Domains](https://mlsanity.com/paper/2604.19341/) (2026)
- [Artificial intelligence for representing and characterizing quantum systems](https://mlsanity.com/paper/2509.04923/) (2025)
- [Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing](https://mlsanity.com/paper/2602.10092/) (2026)
- [Symbolic Analysis of Grover Search Algorithm via Chain-of-Thought Reasoning and Quantum-Native Tokenization](https://mlsanity.com/paper/2505.04880/) (2025)
- [QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation](https://mlsanity.com/paper/2604.08570/) (2026)
- [Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation](https://mlsanity.com/paper/2605.27210/) (2026)
- [Generative AI for Quantum Circuits and Quantum Code: A Technical Review and Taxonomy](https://mlsanity.com/paper/2603.16216/) (2026)
- [QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuits](https://mlsanity.com/paper/2605.30358/) (2026)
- [Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation](https://mlsanity.com/paper/2607.25891/) (2026)
- [LLM-Driven Algorithm Design for Quantum Circuit Synthesis based on Binary Decision Diagrams](https://mlsanity.com/paper/2609.05327/) (2026)
- [Generative Replay Mitigates Sample Starvation in Quantum Architecture Search](https://mlsanity.com/paper/2609.11248/) (2026)
- [Evaluating Verified Autonomy in Quantum Engineering](https://mlsanity.com/paper/2609.17439/) (2026)
- [QEncodeBench: Can Large Language Models Encode Classical Problems into Verified Quantum Oracles?](https://mlsanity.com/paper/2609.20319/) (2026)
- [AG-CoT: Verified Algorithmic Traces for LLM Program Synthesis on Clifford Circuits](https://mlsanity.com/paper/2609.33192/) (2026)
- [QC-Stark: A Multi-Task Benchmark Revealing Capability Dissociations in LLMs Evaluated on Quantum Computing Tasks](https://mlsanity.com/paper/2609.35581/) (2026)

## Abstract (arXiv)

Quantum computing is an emerging field recognized for the significant speedup it offers over classical computing through quantum algorithms. However, designing and implementing quantum algorithms pose challenges due to the complex nature of quantum mechanics and the necessity for precise control over quantum states. Despite the significant advancements in AI, there has been a lack of datasets specifically tailored for this purpose. In this work, we introduce QCircuitBench, the first benchmark dataset designed to evaluate AI's capability in designing and implementing quantum algorithms using quantum programming languages. Unlike using AI for writing traditional codes, this task is fundamentally more complicated due to highly flexible design space. Our key contributions include: 1. A general framework which formulates the key features of quantum algorithm design for Large Language Models. 2. Implementations for quantum algorithms from basic primitives to advanced applications, spanning 3 task suites, 25 algorithms, and 120,290 data points. 3. Automatic validation and verification functions, allowing for iterative evaluation and interactive reasoning without human inspection. 4. Promising potential as a training dataset through preliminary fine-tuning results. We observed several interesting experimental phenomena: LLMs tend to exhibit consistent error patterns, and fine-tuning does not always outperform few-shot learning. In all, QCircuitBench is a comprehensive benchmark for LLM-driven quantum algorithm design, and it reveals limitations of LLMs in this domain.