# An MLCommons Scientific Benchmarks Ontology

arXiv: https://arxiv.org/abs/2511.05614 · HTML: https://mlsanity.com/paper/2511.05614/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 76 references that are in the map.
- First posted on arXiv 6 Nov 2025.

## Builds on (most cited first, 30 of 40)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [The Open Catalyst 2020 (OC20) Dataset and Community Challenges](https://mlsanity.com/paper/2010.09990/) (2020)
- [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://mlsanity.com/paper/2411.04872/) (2024)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning](https://mlsanity.com/paper/2412.00568/) (2024)
- [MLPerf Training Benchmark](https://mlsanity.com/paper/1910.01500/) (2019)
- [The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts](https://mlsanity.com/paper/2206.08917/) (2022)
- [MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation](https://mlsanity.com/paper/2110.01406/) (2021)
- [Better than classical? The subtle art of benchmarking quantum machine learning models](https://mlsanity.com/paper/2403.07059/) (2024)
- [DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics](https://mlsanity.com/paper/1712.03641/) (2017)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [Scientific Machine Learning Benchmarks](https://mlsanity.com/paper/2110.12773/) (2021)
- [ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling](https://mlsanity.com/paper/2307.01909/) (2023)
- [CFDBench: A Large-Scale Benchmark for Machine Learning Methods in Fluid Dynamics](https://mlsanity.com/paper/2310.05963/) (2023)
- [MassSpecGym: A benchmark for the discovery and identification of molecules](https://mlsanity.com/paper/2410.23326/) (2024)
- [CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation](https://mlsanity.com/paper/2410.21611/) (2024)
- [CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning](https://mlsanity.com/paper/2503.13517/) (2025)
- [Domain-Agnostic Molecular Generation with Chemical Feedback](https://mlsanity.com/paper/2301.11259/) (2023)
- [FEABench: Evaluating Language Models on Multiphysics Reasoning Ability](https://mlsanity.com/paper/2504.06260/) (2025)
- [Fathom: Reference Workloads for Modern Deep Learning Methods](https://mlsanity.com/paper/1608.06581/) (2016)
- [MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems](https://mlsanity.com/paper/2110.11466/) (2021)
- [SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models](https://mlsanity.com/paper/2304.11619/) (2023)

## Built on by (most cited first, 2 of 2)

- [AI Benchmark Democratization and Carpentry](https://mlsanity.com/paper/2512.11588/) (2025)
- [Experiments in Agentic AI for Science](https://mlsanity.com/paper/2605.26305/) (2026)

## Abstract (arXiv)

Scientific machine learning research spans diverse domains and data modalities, yet existing benchmark efforts remain siloed and lack standardization. This makes novel and transformative applications of machine learning to critical scientific use-cases more fragmented and less clear in pathways to impact. This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort that extends the MLCommons ecosystem to cover physics, chemistry, materials science, biology, climate science, and more. Building on prior initiatives such as XAI-BENCH, FastML Science Benchmarks, PDEBench, and the SciMLBench framework, our effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy of scientific, application, and system-level benchmarks. New benchmarks can be added through an open submission workflow coordinated by the MLCommons Science Working Group and evaluated against a six-category rating rubric that promotes and identifies high-quality benchmarks, enabling stakeholders to select benchmarks that meet their specific needs. The architecture is extensible, supporting future scientific and AI/ML motifs, and we discuss methods for identifying emerging computing patterns for unique scientific workloads. The MLCommons Science Benchmarks Ontology provides a standardized, scalable foundation for reproducible, cross-domain benchmarking in scientific machine learning. A companion webpage for this work has also been developed as the effort evolves: https://mlcommons-science.github.io/benchmark/