# Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems

arXiv: https://arxiv.org/abs/2205.06502 · HTML: https://mlsanity.com/paper/2205.06502/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Deep Neural Networks for Data-Driven Turbulence Models](https://mlsanity.com/paper/1806.04482/) (2018)
- [A Perspective on Machine Learning Methods in Turbulence Modelling](https://mlsanity.com/paper/2010.12226/) (2020)
- [A Fortran-Keras Deep Learning Bridge for Scientific Computing](https://mlsanity.com/paper/2004.10652/) (2020)
- [A Neural Network based Shock Detection and Localization Approach for Discontinuous Galerkin Methods](https://mlsanity.com/paper/2001.08201/) (2020)
- [FLEXI: A high order discontinuous Galerkin framework for hyperbolic-parabolic conservation laws](https://mlsanity.com/paper/1910.02858/) (2019)
- [Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling](https://mlsanity.com/paper/2104.09355/) (2021)
- [A machine learning framework for LES closure terms](https://mlsanity.com/paper/2010.03030/) (2020)
- [A parallel Fortran framework for neural networks and deep learning](https://mlsanity.com/paper/1902.06714/) (2019)
- [Deploying deep learning in OpenFOAM with TensorFlow](https://mlsanity.com/paper/2012.00900/) (2020)

## Built on by (most cited first, 7 of 7)

- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations](https://mlsanity.com/paper/2206.11038/) (2022)
- [Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning](https://mlsanity.com/paper/2309.06260/) (2023)
- [Harnessing Equivariance: Modeling Turbulence with Graph Neural Networks](https://mlsanity.com/paper/2504.07741/) (2025)
- [Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics](https://mlsanity.com/paper/2402.11515/) (2024)
- [Invariant Control Strategies for Active Flow Control using Graph Neural Networks](https://mlsanity.com/paper/2503.22775/) (2025)
- [Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs](https://mlsanity.com/paper/2604.06497/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) is highly suitable for devising control strategies in the context of dynamical systems. A prominent instance of such a dynamical system is the system of equations governing fluid dynamics. Recent research results indicate that RL-augmented computational fluid dynamics (CFD) solvers can exceed the current state of the art, for example in the field of turbulence modeling. However, while in supervised learning, the training data can be generated a priori in an offline manner, RL requires constant run-time interaction and data exchange with the CFD solver during training. In order to leverage the potential of RL-enhanced CFD, the interaction between the CFD solver and the RL algorithm thus have to be implemented efficiently on high-performance computing (HPC) hardware. To this end, we present Relexi as a scalable RL framework that bridges the gap between machine learning workflows and modern CFD solvers on HPC systems providing both components with its specialized hardware. Relexi is built with modularity in mind and allows easy integration of various HPC solvers by means of the in-memory data transfer provided by the SmartSim library. Here, we demonstrate that the Relexi framework can scale up to hundreds of parallel environment on thousands of cores. This allows to leverage modern HPC resources to either enable larger problems or faster turnaround times. Finally, we demonstrate the potential of an RL-augmented CFD solver by finding a control strategy for optimal eddy viscosity selection in large eddy simulations.