# Data-assimilated model-informed reinforcement learning

arXiv: https://arxiv.org/abs/2506.01755 · HTML: https://mlsanity.com/paper/2506.01755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 65 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 22 of 22)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://mlsanity.com/paper/1707.06203/) (2017)
- [Echo state networks are universal](https://mlsanity.com/paper/1806.00797/) (2018)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [Partially Observed Markov Decision Processes. Problem Sets and Internet Supplement](https://mlsanity.com/paper/1604.08127/) (2016)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics](https://mlsanity.com/paper/2103.03174/) (2021)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning](https://mlsanity.com/paper/2205.00579/) (2022)
- [Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics](https://mlsanity.com/paper/2104.05437/) (2021)
- [Reconstruction, forecasting, and stability of chaotic dynamics from partial data](https://mlsanity.com/paper/2305.15111/) (2023)
- [Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing](https://mlsanity.com/paper/2307.01995/) (2023)
- [Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter](https://mlsanity.com/paper/2306.04315/) (2023)
- [Reinforcement Twinning: from digital twins to model-based reinforcement learning](https://mlsanity.com/paper/2311.03628/) (2023)
- [Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/2401.00916/) (2024)
- [Learning Hidden States in a Chaotic System: A Physics-Informed Echo State Network Approach](https://mlsanity.com/paper/2001.02982/) (2020)
- [Data-Assimilated Model-Based Reinforcement Learning for Partially Observed Chaotic Flows](https://mlsanity.com/paper/2504.16588/) (2025)
- [Control-aware echo state networks (Ca-ESN) for the suppression of extreme events](https://mlsanity.com/paper/2308.03095/) (2023)
- [Online model learning with data-assimilated reservoir computers](https://mlsanity.com/paper/2504.16767/) (2025)
- [Stabilising nontrivial solutions of the generalised Kuramoto-Sivashinsky equation using feedback and optimal control](https://mlsanity.com/paper/1505.06086/) (2015)

## Built on by (most cited first, 1 of 1)

- [Efficient Adaptation of ROMs for Unsteady Flows Using Data Assimilation](https://mlsanity.com/paper/2602.23188/) (2026)

## Abstract (arXiv)

The control of spatio-temporally chaos is challenging because of high dimensionality and unpredictability. Model-free reinforcement learning (RL) discovers optimal control policies by interacting with the system, typically requiring observations of the full physical state. In practice, sensors often provide only partial and noisy measurements (observations) of the system. The objective of this paper is to develop a framework that enables the control of chaotic systems with partial and noisy observability. The proposed method, data-assimilated model-informed reinforcement learning (DA-MIRL), integrates (i) low-order models to approximate high-dimensional dynamics; (ii) sequential data assimilation to correct the model prediction when observations become available; and (iii) an off-policy actor-critic RL algorithm to adaptively learn an optimal control strategy based on the corrected state estimates. We test DA-MIRL on the spatiotemporally chaotic solutions of the Kuramoto-Sivashinsky equation. We estimate the full state of the environment with (i) a physics-based model, here, a coarse-grained model; and (ii) a data-driven model, here, the control-aware echo state network, which is proposed in this paper. We show that DA-MIRL successfully estimates and suppresses the chaotic dynamics of the environment in real time from partial observations and approximate models. This work opens opportunities for the control of partially observable chaotic systems.