# Comparative analysis of machine learning methods for active flow control

arXiv: https://arxiv.org/abs/2202.11664 · HTML: https://mlsanity.com/paper/2202.11664/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 150 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [A Survey of Reinforcement Learning Informed by Natural Language](https://mlsanity.com/paper/1906.03926/) (2019)
- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [A Survey of Optimization Methods from a Machine Learning Perspective](https://mlsanity.com/paper/1906.06821/) (2019)
- [Learning values across many orders of magnitude](https://mlsanity.com/paper/1602.07714/) (2016)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Remember and Forget for Experience Replay](https://mlsanity.com/paper/1807.05827/) (2018)
- [Global optimization of Lipschitz functions](https://mlsanity.com/paper/1703.02628/) (2017)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Controlling Rayleigh-B'enard convection via Reinforcement Learning](https://mlsanity.com/paper/2003.14358/) (2020)
- [Symbolic Regression Methods for Reinforcement Learning](https://mlsanity.com/paper/1903.09688/) (2019)
- [A statistical learning strategy for closed-loop control of fluid flows](https://mlsanity.com/paper/1604.03392/) (2016)
- [Reinforcement Learning via Gaussian Processes with Neural Network Dual Kernels](https://mlsanity.com/paper/2004.05198/) (2020)
- [Combining Bayesian Optimization and Lipschitz Optimization](https://mlsanity.com/paper/1810.04336/) (2018)
- [Efficient Model-Free Reinforcement Learning Using Gaussian Process](https://mlsanity.com/paper/1812.04359/) (2018)

## Built on by (most cited first, 6 of 6)

- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [Deep Learning Closure Models for Large-Eddy Simulation of Flows around Bluff Bodies](https://mlsanity.com/paper/2208.03498/) (2022)
- [Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convection](https://mlsanity.com/paper/2407.21565/) (2024)
- [Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing](https://mlsanity.com/paper/2307.01995/) (2023)
- [How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning](https://mlsanity.com/paper/2304.11526/) (2023)
- [Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics](https://mlsanity.com/paper/2402.11515/) (2024)

## Abstract (arXiv)

Machine learning frameworks such as Genetic Programming (GP) and Reinforcement Learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, bench-marking some of their most representative algorithms against global optimization techniques such as Bayesian Optimization (BO) and Lipschitz global optimization (LIPO). First, we review the general framework of the model-free control problem, bringing together all methods as black-box optimization problems. Then, we test the control algorithms on three test cases. These are (1) the stabilization of a nonlinear dynamical system featuring frequency cross-talk, (2) the wave cancellation from a Burgers' flow and (3) the drag reduction in a cylinder wake flow. We present a comprehensive comparison to illustrate their differences in exploration versus exploitation and their balance between `model capacity' in the control law definition versus `required complexity'. We believe that such a comparison paves the way toward the hybridization of the various methods, and we offer some perspective on their future development in the literature on flow control problems.