# Turbulence control in plane Couette flow using low-dimensional neural ODE-based models and deep reinforcement learning

arXiv: https://arxiv.org/abs/2301.12098 · HTML: https://mlsanity.com/paper/2301.12098/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 69 references that are in the map.
- First posted on arXiv 28 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems](https://mlsanity.com/paper/2203.15706/) (2022)
- [Data-driven discovery of intrinsic dynamics](https://mlsanity.com/paper/2108.05928/) (2021)
- [Deep learning to discover and predict dynamics on an inertial manifold](https://mlsanity.com/paper/2001.04263/) (2019)
- [Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations](https://mlsanity.com/paper/2109.00060/) (2021)
- [Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning](https://mlsanity.com/paper/2205.00579/) (2022)
- [Dynamics of a data-driven low-dimensional model of turbulent minimal Couette flow](https://mlsanity.com/paper/2301.04638/) (2023)

## Built on by (most cited first, 5 of 5)

- [Nonlinear dimensionality reduction then and now: AIMs for dissipative PDEs in the ML era](https://mlsanity.com/paper/2310.15816/) (2023)
- [Gym-preCICE: Reinforcement Learning Environments for Active Flow Control](https://mlsanity.com/paper/2305.02033/) (2023)
- [Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows](https://mlsanity.com/paper/2408.14685/) (2024)
- [Machine-Learned Closure of URANS for Stably Stratified Turbulence: Connecting Physical Timescales & Data Hyperparameters of Deep Time-Series Models](https://mlsanity.com/paper/2404.16141/) (2024)
- [Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics](https://mlsanity.com/paper/2507.21299/) (2025)

## Abstract (arXiv)

The high dimensionality and complex dynamics of turbulent flows remain an obstacle to the discovery and implementation of control strategies. Deep reinforcement learning (RL) is a promising avenue for overcoming these obstacles, but requires a training phase in which the RL agent iteratively interacts with the flow environment to learn a control policy, which can be prohibitively expensive when the environment involves slow experiments or large-scale simulations. We overcome this challenge using a framework we call "DManD-RL" (data-driven manifold dynamics-RL), which generates a data-driven low-dimensional model of our system that we use for RL training. With this approach, we seek to minimize drag in a direct numerical simulation (DNS) of a turbulent minimal flow unit of plane Couette flow at Re=400 using two slot jets on one wall. We obtain, from DNS data with O(105) degrees of freedom, a 25-dimensional DManD model of the dynamics by combining an autoencoder and neural ordinary differential equation. Using this model as the environment, we train an RL control agent, yielding a 440-fold speedup over training on the DNS, with equivalent control performance. The agent learns a policy that laminarizes 84% of unseen DNS test trajectories within 900 time units, significantly outperforming classical opposition control (58%), despite the actuation authority being much more restricted. The agent often achieves laminarization through a counterintuitive strategy that drives the formation of two low-speed streaks, with a spanwise wavelength that is too small to be self-sustaining. The agent demonstrates the same performance when we limit observations to wall shear rate.