# Optimization of geological carbon storage operations with multimodal latent dynamic model and deep reinforcement learning

arXiv: https://arxiv.org/abs/2406.04575 · HTML: https://mlsanity.com/paper/2406.04575/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 32 of its 129 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning](https://mlsanity.com/paper/2206.15378/) (2022)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems](https://mlsanity.com/paper/1908.05823/) (2019)
- [Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media](https://mlsanity.com/paper/1807.00882/) (2018)
- [Real-time high-resolution CO2 geological storage prediction using nested Fourier neural operators](https://mlsanity.com/paper/2210.17051/) (2022)
- [Learning to Accelerate Partial Differential Equations via Latent Global Evolution](https://mlsanity.com/paper/2206.07681/) (2022)
- [Deep Learning of Subsurface Flow via Theory-guided Neural Network](https://mlsanity.com/paper/1911.00103/) (2019)
- [Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method](https://mlsanity.com/paper/2012.06148/) (2020)
- [Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification](https://mlsanity.com/paper/1812.09444/) (2018)
- [Symbolic Pregression: Discovering Physical Laws from Distorted Video](https://mlsanity.com/paper/2005.11212/) (2020)
- [A Deep Learning-Accelerated Data Assimilation and Forecasting Workflow for Commercial-Scale Geologic Carbon Storage](https://mlsanity.com/paper/2105.09468/) (2021)
- [Real-Time Model Calibration with Deep Reinforcement Learning](https://mlsanity.com/paper/2006.04001/) (2020)
- [Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models](https://mlsanity.com/paper/2204.11138/) (2022)
- [Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology](https://mlsanity.com/paper/2203.13375/) (2022)
- [Deep Learning-Accelerated 3D Carbon Storage Reservoir Pressure Forecasting Based on Data Assimilation Using Surface Displacement from InSAR](https://mlsanity.com/paper/2201.08543/) (2022)
- [Identifying optimal cycles in quantum thermal machines with reinforcement-learning](https://mlsanity.com/paper/2108.13525/) (2021)
- [Deep Reinforcement Learning for Constrained Field Development Optimization in Subsurface Two-phase Flow](https://mlsanity.com/paper/2104.00527/) (2021)
- [Convolutional-Recurrent Neural Network Proxy for Robust Optimization and Closed-Loop Reservoir Management](https://mlsanity.com/paper/2203.07524/) (2022)
- [Stochastic optimal well control in subsurface reservoirs using reinforcement learning](https://mlsanity.com/paper/2207.03456/) (2022)

## Abstract (arXiv)

Maximizing storage performance in geological carbon storage (GCS) is crucial for commercial deployment, but traditional optimization demands resource-intensive simulations, posing computational challenges. This study introduces the multimodal latent dynamic (MLD) model, a deep learning framework for fast flow prediction and well control optimization in GCS. The MLD model includes a representation module for compressed latent representations, a transition module for system state evolution, and a prediction module for flow responses. A novel training strategy combining regression loss and joint-embedding consistency loss enhances temporal consistency and multi-step prediction accuracy. Unlike existing models, the MLD supports diverse input modalities, allowing comprehensive data interactions. The MLD model, resembling a Markov decision process (MDP), can train deep reinforcement learning agents, specifically using the soft actor-critic (SAC) algorithm, to maximize net present value (NPV) through continuous interactions. The approach outperforms traditional methods, achieving the highest NPV while reducing computational resources by over 60%. It also demonstrates strong generalization performance, providing improved decisions for new scenarios based on knowledge from previous ones.