# SAIBench: Benchmarking AI for Science

arXiv: https://arxiv.org/abs/2206.05418 · HTML: https://mlsanity.com/paper/2206.05418/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 11 Jun 2022.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [A Differentiable Programming System to Bridge Machine Learning and Scientific Computing](https://mlsanity.com/paper/1907.07587/) (2019)
- [How could Neural Networks understand Programs?](https://mlsanity.com/paper/2105.04297/) (2021)
- [Review: Deep Learning in Electron Microscopy](https://mlsanity.com/paper/2009.08328/) (2020)
- [Machine Learning for Molecular Dynamics on Long Timescales](https://mlsanity.com/paper/1812.07669/) (2018)
- [MLHarness: A Scalable Benchmarking System for MLCommons](https://mlsanity.com/paper/2111.05231/) (2021)

## Built on by (most cited first, 1 of 1)

- [Towards a Benchmark for Scientific Understanding in Humans and Machines](https://mlsanity.com/paper/2304.10327/) (2023)

## Abstract (arXiv)

Scientific research communities are embracing AI-based solutions to target tractable scientific tasks and improve research workflows. However, the development and evaluation of such solutions are scattered across multiple disciplines. We formalize the problem of scientific AI benchmarking, and propose a system called SAIBench in the hope of unifying the efforts and enabling low-friction on-boarding of new disciplines. The system approaches this goal with SAIL, a domain-specific language to decouple research problems, AI models, ranking criteria, and software/hardware configuration into reusable modules. We show that this approach is flexible and can adapt to problems, AI models, and evaluation methods defined in different perspectives. The project homepage is https://www.computercouncil.org/SAIBench