# Reinforcement learning based data assimilation for unknown state model

arXiv: https://arxiv.org/abs/2511.02286 · HTML: https://mlsanity.com/paper/2511.02286/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces](https://mlsanity.com/paper/1905.07357/) (2019)
- [Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization](https://mlsanity.com/paper/2001.06270/) (2020)
- [Data Assimilation: A Mathematical Introduction](https://mlsanity.com/paper/1506.07825/) (2015)
- [Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learning](https://mlsanity.com/paper/1810.10535/) (2018)
- [Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology](https://mlsanity.com/paper/2203.13375/) (2022)
- [Variational Deep Learning for the Identification and Reconstruction of Chaotic and Stochastic Dynamical Systems from Noisy and Partial Observations](https://mlsanity.com/paper/2009.02296/) (2020)
- [Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/2401.00916/) (2024)
- [Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems](https://mlsanity.com/paper/2502.16232/) (2025)
- [RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensembles](https://mlsanity.com/paper/2505.05452/) (2025)

## Built on by (most cited first, 1 of 1)

- [Auto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms](https://mlsanity.com/paper/2603.20891/) (2026)

## Abstract (arXiv)

Data assimilation (DA) has increasingly emerged as a critical tool for state estimation across a wide range of applications. It is significantly challenging when the governing equations of the underlying dynamics are unknown. To this end, various machine learning approaches have been employed to construct a surrogate state transition model in a supervised learning framework, which relies on pre-computed training datasets. However, it is often infeasible to obtain noise-free ground-truth state sequences in practice. To address this challenge, we propose a novel method that integrates reinforcement learning with ensemble-based Bayesian filtering methods, enabling the learning of surrogate state transition model for unknown dynamics directly from noisy observations, without using true state trajectories. Specifically, we treat the process for computing maximum likelihood estimation of surrogate model parameters as a sequential decision-making problem, which can be formulated as a discrete-time Markov decision process (MDP). Under this formulation, learning the surrogate transition model is equivalent to finding an optimal policy of the MDP, which can be effectively addressed using reinforcement learning techniques. Once the model is trained offline, state estimation can be performed in the online stage using filtering methods based on the learned dynamics. The proposed framework accommodates a wide range of observation scenarios, including nonlinear and partially observed measurement models. A few numerical examples demonstrate that the proposed method achieves superior accuracy and robustness in high-dimensional settings.