# QMBench: A Research Level Benchmark for Quantum Materials Research

arXiv: https://arxiv.org/abs/2512.19753 · HTML: https://mlsanity.com/paper/2512.19753/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 19 Dec 2025.

## Builds on (most cited first, 14 of 14)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Accelerating scientific discovery with Co-Scientist](https://mlsanity.com/paper/2502.18864/) (2025)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [Robin: A multi-agent system for automating scientific discovery](https://mlsanity.com/paper/2505.13400/) (2025)
- [ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterization](https://mlsanity.com/paper/2401.06949/) (2024)
- [SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?](https://mlsanity.com/paper/2507.05241/) (2025)
- [Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics](https://mlsanity.com/paper/2502.15815/) (2025)
- [CMT-Benchmark: A Benchmark for Condensed Matter Theory Built by Expert Researchers](https://mlsanity.com/paper/2510.05228/) (2025)

## Built on by (most cited first, 3 of 3)

- [The Agentification of Scientific Research: A Physicist's Perspective](https://mlsanity.com/paper/2604.14718/) (2026)
- [S2SServiceBench: A Multimodal Benchmark for Last-Mile S2S Climate Services](https://mlsanity.com/paper/2602.14017/) (2026)
- [CompMat-Bench: Benchmarking AI Agents for Computational Materials Science](https://mlsanity.com/paper/2610.00636/) (2026)

## Abstract (arXiv)

We introduce QMBench, a comprehensive benchmark designed to evaluate the capability of large language model agents in quantum materials research. This specialized benchmark assesses the model's ability to apply condensed matter physics knowledge and computational techniques such as density functional theory to solve research problems in quantum materials science. QMBench encompasses different domains of the quantum material research, including structural properties, electronic properties, thermodynamic and other properties, symmetry principle and computational methodologies. By providing a standardized evaluation framework, QMBench aims to accelerate the development of an AI scientist capable of making creative contributions to quantum materials research. We expect QMBench to be developed and constantly improved by the research community.