SAIBench: Benchmarking AI for Science
Not on the map. Why?
Abstract (arXiv)
Scientific research communities are embracing AI-based solutions to target tractable scientific tasks and improve research workflows. However, the development and evaluation of such solutions are scattered across multiple disciplines. We formalize the problem of scientific AI benchmarking, and propose a system called SAIBench in the hope of unifying the efforts and enabling low-friction on-boarding of new disciplines. The system approaches this goal with SAIL, a domain-specific language to decouple research problems, AI models, ranking criteria, and software/hardware configuration into reusable modules. We show that this approach is flexible and can adapt to problems, AI models, and evaluation methods defined in different perspectives. The project homepage is https://www.computercouncil.org/SAIBench
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 8 of its 37 references that are in the map. First posted on arXiv 11 Jun 2022.
Builds on
8 of 37 references.
References · 8
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
- 2017VAMPnets: Deep learning of molecular kineticscited by 135
- 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 81
- 2021How could Neural Networks understand Programs?cited by 26
- 2020Review: Deep Learning in Electron Microscopycited by 20
- 2018Machine Learning for Molecular Dynamics on Long Timescalescited by 4
- 2021MLHarness: A Scalable Benchmarking System for MLCommonscited by 1
Built on by
Citing papers per year: 2023: 1.
1 paper.