AI-coupled HPC Workflows
Not on the map. Why?
Abstract (arXiv)
Increasingly, scientific discovery requires sophisticated and scalable workflows. Workflows have become the ``new applications,'' wherein multi-scale computing campaigns comprise multiple and heterogeneous executable tasks. In particular, the introduction of AI/ML models into the traditional HPC workflows has been an enabler of highly accurate modeling, typically reducing computational needs compared to traditional methods. This chapter discusses various modes of integrating AI/ML models to HPC computations, resulting in diverse types of AI-coupled HPC workflows. The increasing need of coupling AI/ML and HPC across scientific domains is motivated, and then exemplified by a number of production-grade use cases for each mode. We additionally discuss the primary challenges of extreme-scale AI-coupled HPC campaigns -- task heterogeneity, adaptivity, performance -- and several framework and middleware solutions which aim to address them. While both HPC workflow and AI/ML computing paradigms are independently effective, we highlight how their integration, and ultimate convergence, is leading to significant improvements in scientific performance across a range of domains, ultimately resulting in scientific explorations otherwise unattainable.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 13 of its 43 references that are in the map. First posted on arXiv 24 Aug 2022.
Builds on
13 of 43 references.
References · 13
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
- 2016Theory-guided Data Science: A New Paradigm for Scientific Discovery from Datacited by 171
- 2017DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamicscited by 79
- 2020Building high accuracy emulators for scientific simulations with deep neural architecture searchcited by 36
- 2019DeepDriveMD: Deep-Learning Driven Adaptive Molecular Simulations for Protein Foldingcited by 15
- 2019Scalable Reinforcement-Learning-Based Neural Architecture Search for Cancer Deep Learning Researchcited by 10
- 2021Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computingcited by 10
- 2019Learning Everywhere: A Taxonomy for the Integration of Machine Learning and Simulationscited by 7
- 2019Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computationcited by 6
- 2020IMPECCABLE: Integrated Modeling PipelinE for COVID Cure by Assessing Better LEadscited by 6
- 2020Targeting SARS-CoV-2 with AI- and HPC-enabled Lead Generation: A First Data Releasecited by 5
- 2019Understanding ML driven HPC: Applications and Infrastructurecited by 2
1 more
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.