# Enabling Machine Learning-Ready HPC Ensembles with Merlin

arXiv: https://arxiv.org/abs/1912.02892 · HTML: https://mlsanity.com/paper/1912.02892/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2019.

## Built on by (most cited first, 7 of 7)

- [Training Deep Surrogate Models with Large Scale Online Learning](https://mlsanity.com/paper/2306.16133/) (2023)
- [Machine Learning Visualization Tool for Exploring Parameterized Hydrodynamics](https://mlsanity.com/paper/2406.15509/) (2024)
- [Using Conservation Laws to Infer Deep Learning Model Accuracy of Richtmyer-meshkov Instabilities](https://mlsanity.com/paper/2208.11477/) (2022)
- [Multi-Agent Collaboration for Automated Design Exploration on High Performance Computing Systems](https://mlsanity.com/paper/2603.11515/) (2026)
- [Asynchronous Execution of Heterogeneous Tasks in ML-driven HPC Workflows](https://mlsanity.com/paper/2208.11069/) (2022)
- [Machine Learning-driven Multiscale MD Workflows: The Mini-MuMMI Experience](https://mlsanity.com/paper/2507.07352/) (2025)
- [What Artificial Intelligence can do for High-Performance Computing systems?](https://mlsanity.com/paper/2602.00014/) (2026)

## Abstract (arXiv)

With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. In addition to its design, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.