# Optimization of Pressure Management Strategies for Geological CO2 Sequestration Using Surrogate Model-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2403.07360 · HTML: https://mlsanity.com/paper/2403.07360/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2024.

## Built on by (most cited first, 5 of 5)

- [TD3 Based Collision Free Motion Planning for Robot Navigation](https://mlsanity.com/paper/2405.15460/) (2024)
- [Utilizing Deep Learning to Optimize Software Development Processes](https://mlsanity.com/paper/2404.13630/) (2024)
- [Autonomous Navigation of Unmanned Vehicle Through Deep Reinforcement Learning](https://mlsanity.com/paper/2407.18962/) (2024)
- [Research and Design on Intelligent Recognition of Unordered Targets for Robots Based on Reinforcement Learning](https://mlsanity.com/paper/2503.07340/) (2025)
- [Multi-Step Embed to Control: A Novel Deep Learning-based Approach for Surrogate Modelling in Reservoir Simulation](https://mlsanity.com/paper/2409.09920/) (2024)

## Abstract (arXiv)

Injecting greenhouse gas into deep underground reservoirs for permanent storage can inadvertently lead to fault reactivation, caprock fracturing and greenhouse gas leakage when the injection-induced stress exceeds the critical threshold. Extraction of pre-existing fluids at various stages of injection process, referred as pressure management, can mitigate associated risks and lessen environmental impact. However, identifying optimal pressure management strategies typically requires thousands of full-order simulations due to the need for function evaluations, making the process computationally prohibitive. This paper introduces a novel surrogate model-based reinforcement learning method for devising optimal pressure management strategies for geological CO2 sequestration efficiently. Our approach comprises two steps. Firstly, a surrogate model is developed through the embed to control method, which employs an encoder-transition-decoder structure to learn latent dynamics. Leveraging this proxy model, reinforcement learning is utilized to find an optimal strategy that maximizes economic benefits while satisfying various control constraints. The reinforcement learning agent receives the latent state space representation and immediate reward tailored for CO2 sequestration and choose real-time controls which are subject to predefined engineering constraints in order to maximize the long-term cumulative rewards. To demonstrate its effectiveness, this framework is applied to a compositional simulation model where CO2 is injected into saline aquifer. The results reveal that our surrogate model-based reinforcement learning approach significantly optimizes CO2 sequestration strategies, leading to notable economic gains compared to baseline scenarios.