# How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2304.11526 · HTML: https://mlsanity.com/paper/2304.11526/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 84 references that are in the map.
- First posted on arXiv 23 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Explainable Deep Reinforcement Learning: State of the Art and Challenges](https://mlsanity.com/paper/2301.09937/) (2023)
- [Data-Driven Model Predictive Control using Interpolated Koopman Generators](https://mlsanity.com/paper/2003.07094/) (2020)
- [Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows](https://mlsanity.com/paper/2006.02979/) (2020)
- [Applying Machine Learning to Study Fluid Mechanics](https://mlsanity.com/paper/2110.02083/) (2021)
- [Comparative analysis of machine learning methods for active flow control](https://mlsanity.com/paper/2202.11664/) (2022)
- [From bypass transition to flow control and data-driven turbulence modeling: An input-output viewpoint](https://mlsanity.com/paper/2003.10104/) (2020)

## Built on by (most cited first, 5 of 5)

- [Wavelet Diffusion Neural Operator](https://mlsanity.com/paper/2412.04833/) (2024)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [CL-DiffPhyCon: Closed-loop Diffusion Control of Complex Physical Systems](https://mlsanity.com/paper/2408.03124/) (2024)
- [Multi-modal Policies with Physics-informed Representations in Complex Fluid Environments](https://mlsanity.com/paper/2410.15250/) (2024)
- [Towards an AI Fluid Scientist: LLM-Powered Scientific Discovery in Experimental Fluid Mechanics](https://mlsanity.com/paper/2512.04716/) (2025)

## Abstract (arXiv)

Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems.