# Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows

arXiv: https://arxiv.org/abs/2504.16588 · HTML: https://mlsanity.com/paper/2504.16588/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 6 of 6)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement](https://mlsanity.com/paper/1604.08127/) (2016)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics](https://mlsanity.com/paper/2103.03174/) (2021)
- [Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter](https://mlsanity.com/paper/2306.04315/) (2023)
- [Control-aware echo state networks (Ca-ESN) for the suppression of extreme events](https://mlsanity.com/paper/2308.03095/) (2023)

## Built on by (most cited first, 3 of 3)

- [PAINT: Parallel-in-time Neural Twins for Dynamical System Reconstruction](https://mlsanity.com/paper/2510.16004/) (2025)
- [Data-assimilated model-informed reinforcement learning](https://mlsanity.com/paper/2506.01755/) (2025)
- [Equation-Free Digital Twins for Nonlinear Structural Dynamics](https://mlsanity.com/paper/2605.00950/) (2026)

## Abstract (arXiv)

The goal of many applications in energy and transport sectors is to control turbulent flows. However, because of chaotic dynamics and high dimensionality, the control of turbulent flows is exceedingly difficult. Model-free reinforcement learning (RL) methods can discover optimal control policies by interacting with the environment, but they require full state information, which is often unavailable in experimental settings. We propose a data-assimilated model-based RL (DA-MBRL) framework for systems with partial observability and noisy measurements. Our framework employs a control-aware Echo State Network for data-driven prediction of the dynamics, and integrates data assimilation with an Ensemble Kalman Filter for real-time state estimation. An off-policy actor-critic algorithm is employed to learn optimal control strategies from state estimates. The framework is tested on the Kuramoto-Sivashinsky equation, demonstrating its effectiveness in stabilizing a spatiotemporally chaotic flow from noisy and partial measurements.