mlsanityData to 7 Oct 2026

Deep Reinforcement Learning for Constrained Field Development Optimization in Subsurface Two-phase Flow

arXiv 2104.00527 ↗First posted 31 Mar 2021cs.LG cs.AI math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

We present a deep reinforcement learning-based artificial intelligence agent that could provide optimized development plans given a basic description of the reservoir and rock/fluid properties with minimal computational cost. This artificial intelligence agent, comprising of a convolutional neural network, provides a mapping from a given state of the reservoir model, constraints, and economic condition to the optimal decision (drill/do not drill and well location) to be taken in the next stage of the defined sequential field development planning process. The state of the reservoir model is defined using parameters that appear in the governing equations of the two-phase flow. A feedback loop training process referred to as deep reinforcement learning is used to train an artificial intelligence agent with such a capability. The training entails millions of flow simulations with varying reservoir model descriptions (structural, rock and fluid properties), operational constraints, and economic conditions. The parameters that define the reservoir model, operational constraints, and economic conditions are randomly sampled from a defined range of applicability. Several algorithmic treatments are introduced to enhance the training of the artificial intelligence agent. After appropriate training, the artificial intelligence agent provides an optimized field development plan instantly for new scenarios within the defined range of applicability. This approach has advantages over traditional optimization algorithms (e.g., particle swarm optimization, genetic algorithm) that are generally used to find a solution for a specific field development scenario and typically not generalizable to different scenarios.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 54 references that are in the map. First posted on arXiv 31 Mar 2021.

Builds on

10 of 54 references.

References · 10

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  7. 2020A Closer Look at Invalid Action Masking in Policy Gradient Algorithmscited by 160
  8. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  9. 2020Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Networkcited by 17
  10. 2020Deep Reinforcement Learning for Field Development Optimizationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 0; 2024: 1; 2025: 0; 2026: 1.

5 papers.

  1. 2022Deep reinforcement learning for optimal well control in subsurface systems with uncertain geologycited by 9
  2. 2022Robust optimal well control using an adaptive multi-grid reinforcement learning frameworkcited by 2
  3. 2022Multi-Asset Closed-Loop Reservoir Management Using Deep Reinforcement Learningcited by 1
  4. 2024Optimization of geological carbon storage operations with multimodal latent dynamic model and deep reinforcement learningcited by 0
  5. 2026Closed-Loop CO2 Storage Control With History-Based Reinforcement Learning and Latent Model-Based Adaptationcited by 0