mlsanityData to 14 Oct 2026

Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows

arXiv 2504.16588 ↗First posted 23 Apr 2025eess.SY cs.LG cs.SYcited by 3

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

The goal of many applications in energy and transport sectors is to control turbulent flows. However, because of chaotic dynamics and high dimensionality, the control of turbulent flows is exceedingly difficult. Model-free reinforcement learning (RL) methods can discover optimal control policies by interacting with the environment, but they require full state information, which is often unavailable in experimental settings. We propose a data-assimilated model-based RL (DA-MBRL) framework for systems with partial observability and noisy measurements. Our framework employs a control-aware Echo State Network for data-driven prediction of the dynamics, and integrates data assimilation with an Ensemble Kalman Filter for real-time state estimation. An off-policy actor-critic algorithm is employed to learn optimal control strategies from state estimates. The framework is tested on the Kuramoto-Sivashinsky equation, demonstrating its effectiveness in stabilizing a spatiotemporally chaotic flow from noisy and partial measurements.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 13 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

6 of 13 references.

References · 6

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2016Partially Observed Markov Decision Processes. Problem Sets and Internet Supplementcited by 34
  3. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  4. 2021Robust Optimization and Validation of Echo State Networks for learning chaotic dynamicscited by 27
  5. 2023Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filtercited by 5
  6. 2023Control-aware echo state networks (Ca-ESN) for the suppression of extreme eventscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025PAINT: Parallel-in-time Neural Twins for Dynamical System Reconstructioncited by 2
  2. 2025Data-assimilated model-informed reinforcement learningcited by 1
  3. 2026Equation-Free Digital Twins for Nonlinear Structural Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning
  2. 2026Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extraction
  3. 2025RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensembles
  4. 2025Reinforcement learning based data assimilation for unknown state model
  5. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
  6. 2025Improving the Robustness of Control of Chaotic Convective Flows with Domain-Informed Reinforcement Learning
  7. 2025Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime
  8. 2025Domain Adaptation of Drag Reduction Policy to Partial Measurements