mlsanityData to 14 Oct 2026

Data-assimilated model-informed reinforcement learning

arXiv 2506.01755 ↗First posted 2 Jun 2025eess.SY cs.LG cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

The control of spatio-temporally chaos is challenging because of high dimensionality and unpredictability. Model-free reinforcement learning (RL) discovers optimal control policies by interacting with the system, typically requiring observations of the full physical state. In practice, sensors often provide only partial and noisy measurements (observations) of the system. The objective of this paper is to develop a framework that enables the control of chaotic systems with partial and noisy observability. The proposed method, data-assimilated model-informed reinforcement learning (DA-MIRL), integrates (i) low-order models to approximate high-dimensional dynamics; (ii) sequential data assimilation to correct the model prediction when observations become available; and (iii) an off-policy actor-critic RL algorithm to adaptively learn an optimal control strategy based on the corrected state estimates. We test DA-MIRL on the spatiotemporally chaotic solutions of the Kuramoto-Sivashinsky equation. We estimate the full state of the environment with (i) a physics-based model, here, a coarse-grained model; and (ii) a data-driven model, here, the control-aware echo state network, which is proposed in this paper. We show that DA-MIRL successfully estimates and suppresses the chaotic dynamics of the environment in real time from partial observations and approximate models. This work opens opportunities for the control of partially observable chaotic systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 65 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

22 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows23 Apr
  2. 2024Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning1 Jan

Other references · 20

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  4. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 140
  5. 2018Echo state networks are universalcited by 87
  6. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  7. 2016Partially Observed Markov Decision Processes. Problem Sets and Internet Supplementcited by 34
  8. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  9. 2021Robust Optimization and Validation of Echo State Networks for learning chaotic dynamicscited by 27
  10. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 23
  11. 2022Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learningcited by 17
  12. 2021Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamicscited by 11
8 more
  1. 2023Reconstruction, forecasting, and stability of chaotic dynamics from partial datacited by 8
  2. 2023Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensingcited by 7
  3. 2023Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filtercited by 5
  4. 2023Reinforcement Twinning: from digital twins to model-based reinforcement learningcited by 5
  5. 2020Learning Hidden States in a Chaotic System: A Physics-Informed Echo State Network Approachcited by 3
  6. 2023Control-aware echo state networks (Ca-ESN) for the suppression of extreme eventscited by 2
  7. 2025Online model learning with data-assimilated reservoir computerscited by 2
  8. 2015Stabilising nontrivial solutions of the generalised Kuramoto-Sivashinsky equation using feedback and optimal controlcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Efficient Adaptation of ROMs for Unsteady Flows Using Data Assimilationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reinforcement learning based data assimilation for unknown state model
  2. 2026Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extraction
  3. 2025Policy-Driven World Model Adaptation for Robust Offline Model-based Reinforcement Learning
  4. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
  5. 2025RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensembles
  6. 2024Offline Model-Based Reinforcement Learning with Anti-Exploration
  7. 2024Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability
  8. 2026Offline Policy Optimization with Posterior Sampling