# QCBench: Evaluating Large Language Models on Domain-Specific Quantitative Chemistry

arXiv: https://arxiv.org/abs/2508.01670 · HTML: https://mlsanity.com/paper/2508.01670/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 8 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Builds on (most cited first, 7 of 7)

- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research](https://mlsanity.com/paper/2308.13149/) (2023)
- [Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models](https://mlsanity.com/paper/2305.15074/) (2023)
- [A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges](https://mlsanity.com/paper/2412.11936/) (2024)
- [Developing ChemDFM as a large language foundation model for chemistry](https://mlsanity.com/paper/2401.14818/) (2024)
- [SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark](https://mlsanity.com/paper/2402.05138/) (2024)
- [Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics](https://mlsanity.com/paper/2502.15815/) (2025)

## Built on by (most cited first, 8 of 8)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [SUPERChem: A Multimodal Reasoning Benchmark in Chemistry](https://mlsanity.com/paper/2512.01274/) (2025)
- [PolyReal: A Benchmark for Real-World Polymer Science Workflows](https://mlsanity.com/paper/2604.02934/) (2026)
- [A Task-Centric Ontology and Deterministic Domain Rules as a Verifiable Core for AI-Assisted Chemistry Problem Solving](https://mlsanity.com/paper/2608.26164/) (2026)
- [S3-Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants](https://mlsanity.com/paper/2609.09852/) (2026)
- [ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning](https://mlsanity.com/paper/2610.17183/) (2026)
- [LabOPBench: A Multimodal Benchmark for Evidence-Grounded Laboratory Decision Making](https://mlsanity.com/paper/2610.19780/) (2026)
- [ChemArgus: Automated Scientific Diagnostics for Lab-Grade Multimodal Chemistry Reasoning: A Frontier Benchmark](https://mlsanity.com/paper/2610.20012/) (2026)

## Abstract (arXiv)

Quantitative chemistry is central to modern chemical research, yet the ability of large language models (LLMs) to perform its rigorous, step-by-step calculations remains underexplored. To fill this blank, we propose QCBench, a Quantitative Chemistry oriented benchmark comprising 350 computational chemistry problems across 7 chemistry subfields, which contains analytical chemistry, bio/organic chemistry, general chemistry, inorganic chemistry, physical chemistry, polymer chemistry and quantum chemistry. To systematically evaluate the mathematical reasoning abilities of large language models (LLMs), they are categorized into three tiers: easy, medium, and difficult. Each problem, rooted in realistic chemical scenarios, is structured to prevent heuristic shortcuts and demand explicit numerical reasoning. QCBench enables fine-grained diagnosis of computational weaknesses, reveals model-specific limitations across difficulty levels, and lays the groundwork for future improvements such as domain-adaptive fine-tuning or multi-modal integration. Evaluations on 24 LLMs demonstrate a consistent performance degradation with increasing task complexity, highlighting the current gap between language fluency and scientific computation accuracy. Code for QCBench is available at https://github.com/jiaqingxie/QCBench.