# Quantum-Audit: Evaluating the Reasoning Limits of LLMs on Quantum Computing

arXiv: https://arxiv.org/abs/2602.10092 · HTML: https://mlsanity.com/paper/2602.10092/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 235 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 15 of 15)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models](https://mlsanity.com/paper/2410.07985/) (2024)
- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Better than classical? The subtle art of benchmarking quantum machine learning models](https://mlsanity.com/paper/2403.07059/) (2024)
- [MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data](https://mlsanity.com/paper/2406.18321/) (2024)
- [Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics](https://mlsanity.com/paper/2502.15815/) (2025)
- [Towards Robust Legal Reasoning: Harnessing Logical LLMs in Law](https://mlsanity.com/paper/2502.17638/) (2025)
- [QCircuitBench: A Large-Scale Dataset for Benchmarking Quantum Algorithm Design](https://mlsanity.com/paper/2410.07961/) (2024)
- [QuanBench: Benchmarking Quantum Code Generation with Large Language Models](https://mlsanity.com/paper/2510.16779/) (2025)
- [GroverGPT: A Large Language Model with 8 Billion Parameters for Quantum Searching](https://mlsanity.com/paper/2501.00135/) (2024)
- [Security Attacks Abusing Pulse-level Quantum Circuits](https://mlsanity.com/paper/2406.05941/) (2024)
- [The Reliability of LLMs for Medical Diagnosis: An Examination of Consistency, Manipulation, and Contextual Awareness](https://mlsanity.com/paper/2503.10647/) (2025)
- [Resource-efficient equivariant quantum convolutional neural networks](https://mlsanity.com/paper/2410.01252/) (2024)
- [QubitHammer: Remotely Inducing Qubit State Change on Superconducting Quantum Computers](https://mlsanity.com/paper/2504.07875/) (2025)
- [Can LLMs Help Allocate Public Health Resources? A Case Study on Childhood Lead Testing](https://mlsanity.com/paper/2511.18239/) (2025)

## Built on by (most cited first, 5 of 5)

- [gwBenchmarks: Stress-Testing LLM Agents on High-Precision Gravitational Wave Astronomy](https://mlsanity.com/paper/2605.11269/) (2026)
- [Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluation](https://mlsanity.com/paper/2605.27210/) (2026)
- [QFoldAgent: An Autonomous Quantum Optimization Multi-Agent System for Protein Structure Prediction](https://mlsanity.com/paper/2607.22549/) (2026)
- [QEncodeBench: Can Large Language Models Encode Classical Problems into Verified Quantum Oracles?](https://mlsanity.com/paper/2609.20319/) (2026)
- [QC-Stark: A Multi-Task Benchmark Revealing Capability Dissociations in LLMs Evaluated on Quantum Computing Tasks](https://mlsanity.com/paper/2609.35581/) (2026)

## Abstract (arXiv)

Language models have become practical tools for quantum computing education and research, from summarizing technical papers to explaining theoretical concepts and answering questions about recent developments in the field. While existing benchmarks evaluate quantum code generation and circuit design, their understanding of quantum computing concepts has not been systematically measured. Quantum-Audit addresses this gap with 2,700 questions covering core quantum computing topics. We evaluate 26 models from leading organizations. Our benchmark comprises 1,000 expert-written questions, 1,000 questions extracted from research papers using LLMs and validated by experts, plus an additional 700 questions including 350 open-ended questions and 350 questions with false premises to test whether models can correct erroneous assumptions. Human participants scored between 23% and 86%, with experts averaging 74%. Top-performing models exceeded the expert average, with Claude Opus 4.5 reaching 84% accuracy, though top models showed an average 12-point accuracy drop on expert-written questions compared to LLM-generated ones. Performance declined further on advanced topics, dropping to 73% on security questions. Additionally, models frequently accepted and reinforced false premises embedded in questions instead of identifying them, with accuracy below 66% on these critical reasoning tasks.