# A Two-Stage, Model-Based Reinforcement Learning Approach for Active Flow Control of Bluff Body Wakes

arXiv: https://arxiv.org/abs/2609.08436 · HTML: https://mlsanity.com/paper/2609.08436/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 18 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 2 of 2)

- [An Optimal Solution to Infinite Horizon Nonlinear Control Problems](https://mlsanity.com/paper/2304.00375/) (2023)
- [An Optimal Solution to Infinite Horizon Nonholonomic and Discounted Nonlinear Control Problems](https://mlsanity.com/paper/2403.16979/) (2024)

## Abstract (arXiv)

This paper develops a data-driven, output-feedback approach to the infinite-horizon optimal control of high-dimensional nonlinear systems with unknown and unstable equilibria, using sparse partial observations. The approach builds on the transfer-plus-regulation decomposition of the infinite-horizon problem: a finite-horizon nonlinear transfer drives the system into a region where the dynamics are well-approximated by a linear model about the unknown operating point, and an infinite-horizon linear regulator identified within that region completes stabilization. We extend this framework to the partially observed setting by combining an ARMA-based information-state construction with a two-stage control architecture: an iterative linear quadratic regulator (iLQR) approach on the information state drives the system to the equilibrium neighborhood, discovered implicitly without prior knowledge of the target, and a locally identified time-invariant ARMA model provides the infinite-horizon regulator for asymptotic stabilization. The method requires no adjoint solver, reduced-order model, or full-state access. We validate the approach on high-fidelity Navier-Stokes simulations of the cylinder wake at Re=100 using only eight surface pressure sensors, an order of magnitude fewer than recent model-based RL methods. The controller achieves complete suppression of vortex-shedding-induced lift oscillations and a 44% reduction in total drag relative to the uncontrolled baseline.