# Stabilising viscous extensional flows using Reinforcement Learning

arXiv: https://arxiv.org/abs/2110.14677 · HTML: https://mlsanity.com/paper/2110.14677/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 52 references that are in the map.
- First posted on arXiv 27 Oct 2021.

## Builds on (most cited first, 7 of 7)

- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [Flow Navigation by Smart Microswimmers via Reinforcement Learning](https://mlsanity.com/paper/1701.08848/) (2017)
- [Direct shape optimization through deep reinforcement learning](https://mlsanity.com/paper/1908.09885/) (2019)
- [Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learning](https://mlsanity.com/paper/1711.05826/) (2017)
- [Learning to swim in potential flow](https://mlsanity.com/paper/2009.14280/) (2020)
- [A statistical learning strategy for closed-loop control of fluid flows](https://mlsanity.com/paper/1604.03392/) (2016)

## Abstract (arXiv)

The four-roll mill, wherein four identical cylinders undergo rotation of identical magnitude but alternate signs, was originally proposed by GI Taylor to create local extensional flows and study their ability to deform small liquid drops. Since an extensional flow has an unstable eigendirection, a drop located at the flow stagnation point will have a tendency to escape. This unstable dynamics can however be stabilised using, e.g., a modulation of the rotation rates of the cylinders. Here we use Reinforcement Learning, a branch of Machine Learning devoted to the optimal selection of actions based on cumulative rewards, in order to devise a stabilisation algorithm for the four-roll mill flow. The flow is modelled as the linear superposition of four two-dimensional rotlets and the drop is treated as a rigid spherical particle smaller than all other length scales in the problem. Unlike previous attempts to devise control, we take a probabilistic approach whereby speed adjustments are drawn from a probability density function whose shape is improved over time via a form of gradient ascent know as Actor-Critic method. With enough training, our algorithm is able to precisely control the drop and keep it close to the stagnation point for as long as needed. We explore the impact of the physical and learning parameters on the effectiveness of the control and demonstrate the robustness of the algorithm against thermal noise. We finally show that Reinforcement Learning can provide a control algorithm effective for all initial positions and that can be adapted to limit the magnitude of the flow extension near the position of the drop.