# RL-DAUNCE: Reinforcement Learning-Driven Data Assimilation with Uncertainty-Aware Constrained Ensembles

arXiv: https://arxiv.org/abs/2505.05452 · HTML: https://mlsanity.com/paper/2505.05452/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 97 references that are in the map.
- First posted on arXiv 8 May 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 22 of 22)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Safe Policies for Reinforcement Learning via Primal-Dual Methods](https://mlsanity.com/paper/1911.09101/) (2019)
- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems](https://mlsanity.com/paper/1908.05823/) (2019)
- [Increasing the Action Gap: New Operators for Reinforcement Learning](https://mlsanity.com/paper/1512.04860/) (2015)
- [Using machine learning to correct model error in data assimilation and forecast applications](https://mlsanity.com/paper/2010.12605/) (2020)
- [Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization](https://mlsanity.com/paper/2001.06270/) (2020)
- [A Brief Introduction to Manifold Optimization](https://mlsanity.com/paper/1906.05450/) (2019)
- [Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation](https://mlsanity.com/paper/2007.07383/) (2020)
- [Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations](https://mlsanity.com/paper/2108.03561/) (2021)
- [Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation](https://mlsanity.com/paper/2109.12269/) (2021)
- [Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems](https://mlsanity.com/paper/2206.04811/) (2022)
- [Data Assimilation Networks](https://mlsanity.com/paper/2010.09694/) (2020)
- [CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation](https://mlsanity.com/paper/2410.20072/) (2024)
- [A multi-model ensemble Kalman filter for data assimilation and forecasting](https://mlsanity.com/paper/2202.02272/) (2022)
- [Data Assimilation in Chaotic Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/2401.00916/) (2024)

## Built on by (most cited first, 2 of 2)

- [Reinforcement learning based data assimilation for unknown state model](https://mlsanity.com/paper/2511.02286/) (2025)
- [Omega: Operator-based Mixture Ensemble for Generative Assimilation](https://mlsanity.com/paper/2606.20920/) (2026)

## Abstract (arXiv)

Machine learning has become a powerful tool for enhancing data assimilation. While supervised learning remains the standard method, reinforcement learning (RL) offers unique advantages through its sequential decision-making framework, which naturally fits the iterative nature of data assimilation by dynamically balancing model forecasts with observations. We develop RL-DAUNCE, a new RL-based method that enhances data assimilation with physical constraints through three key aspects. First, RL-DAUNCE inherits the computational efficiency of machine learning while it uniquely structures its agents to mirror ensemble members in conventional data assimilation methods. Second, RL-DAUNCE emphasizes uncertainty quantification by advancing multiple ensemble members, moving beyond simple mean-state optimization. Third, RL-DAUNCE's ensemble-as-agents design facilitates the enforcement of physical constraints during the assimilation process, which is crucial to improving the state estimation and subsequent forecasting. A primal-dual optimization strategy is developed to enforce constraints, which dynamically penalizes the reward function to ensure constraint satisfaction throughout the learning process. Also, state variable bounds are respected by constraining the RL action space. Together, these features ensure physical consistency without sacrificing efficiency. RL-DAUNCE is applied to the Madden-Julian Oscillation, an intermittent atmospheric phenomenon characterized by strongly non-Gaussian features and multiple physical constraints. RL-DAUNCE outperforms the standard ensemble Kalman filter (EnKF), which fails catastrophically due to the violation of physical constraints. Notably, RL-DAUNCE matches the performance of constrained EnKF, particularly in recovering intermittent signals, capturing extreme events, and quantifying uncertainties, while requiring substantially less computational effort.