SAIH: A Scalable Evaluation Methodology for Understanding AI Performance Trend on HPC Systems
Not on the map. Why?
Abstract (arXiv)
Novel artificial intelligence (AI) technology has expedited various scientific research, e.g., cosmology, physics and bioinformatics, inevitably becoming a significant category of workload on high performance computing (HPC) systems. Existing AI benchmarks tend to customize well-recognized AI applications, so as to evaluate the AI performance of HPC systems under predefined problem size, in terms of datasets and AI models. Due to lack of scalability on the problem size, static AI benchmarks might be under competent to help understand the performance trend of evolving AI applications on HPC systems, in particular, the scientific AI applications on large-scale systems. In this paper, we propose a scalable evaluation methodology (SAIH) for analyzing the AI performance trend of HPC systems with scaling the problem sizes of customized AI applications. To enable scalability, SAIH builds a set of novel mechanisms for augmenting problem sizes. As the data and model constantly scale, we can investigate the trend and range of AI performance on HPC systems, and further diagnose system bottlenecks. To verify our methodology, we augment a cosmological AI application to evaluate a real HPC system equipped with GPUs as a case study of SAIH.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 39 references that are in the map. First posted on arXiv 7 Dec 2022.
Builds on
15 of 39 references.
References · 15
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2017Progressive Neural Architecture Searchcited by 483
- 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 169
- 2019MLPerf Training Benchmarkcited by 114
- 2018Exascale Deep Learning for Climate Analyticscited by 28
- 2021MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systemscited by 15
- 2018CosmoFlow: Using Deep Learning to Learn the Universe at Scalecited by 14
- 2017Estimating Cosmological Parameters from the Dark Matter Distributioncited by 11
- 2019Scalable Reinforcement-Learning-Based Neural Architecture Search for Cancer Deep Learning Researchcited by 10
- 2019A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learningcited by 9
- 2018TBD: Benchmarking and Analyzing Deep Neural Network Trainingcited by 6
3 more
- 2019Comprehensive Evaluation of Deep Learning Architectures for Prediction of DNA/RNA Sequence Binding Specificitiescited by 4
- 2019AI Matrix: A Deep Learning Benchmark for Alibaba Data Centerscited by 4
- 2017Deep Learning at 15PF: Supervised and Semi-Supervised Classification for Scientific Datacited by 3
Built on by
Not cited yet.