# Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics

arXiv: https://arxiv.org/abs/2402.11515 · HTML: https://mlsanity.com/paper/2402.11515/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 77 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Controlling Rayleigh-B'enard convection via Reinforcement Learning](https://mlsanity.com/paper/2003.14358/) (2020)
- [LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations](https://mlsanity.com/paper/1808.07903/) (2018)
- [Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems](https://mlsanity.com/paper/2205.06502/) (2022)
- [Comparative analysis of machine learning methods for active flow control](https://mlsanity.com/paper/2202.11664/) (2022)
- [Active Noise Control in The New Century: The Role and Prospect of Signal Processing](https://mlsanity.com/paper/2306.01425/) (2023)

## Built on by (most cited first, 1 of 1)

- [Physics-integrated neural differentiable modeling for immersed boundary systems](https://mlsanity.com/paper/2603.16277/) (2026)

## Abstract (arXiv)

Deep Reinforcement Learning (DRL) has emerged as a promising approach for handling highly dynamic and nonlinear Active Flow Control (AFC) problems. However, the computational cost associated with training DRL models presents a significant performance bottleneck. To address this challenge and enable efficient scaling on high-performance computing architectures, this study focuses on optimizing DRL-based algorithms in parallel settings. We validate an existing state-of-the-art DRL framework used for AFC problems and discuss its efficiency bottlenecks. Subsequently, by deconstructing the overall framework and conducting extensive scalability benchmarks for individual components, we investigate various hybrid parallelization configurations and propose efficient parallelization strategies. Moreover, we refine input/output (I/O) operations in multi-environment DRL training to tackle critical overhead associated with data movement. Finally, we demonstrate the optimized framework for a typical AFC problem where near-linear scaling can be obtained for the overall framework. We achieve a significant boost in parallel efficiency from around 49% to approximately 78%, and the training process is accelerated by approximately 47 times using 60 central processing unit (CPU) cores. These findings are expected to provide valuable insights for further advancements in DRL-based AFC studies. Consequently, it continues to be a prominent and actively studied problem of significant interest.