# Optimal Sequential Decision-Making in Geosteering: A Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2310.04772 · HTML: https://mlsanity.com/paper/2310.04772/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology](https://mlsanity.com/paper/2203.13375/) (2022)
- [A Decision Support System for Multi-target Geosteering](https://mlsanity.com/paper/1903.03933/) (2019)
- [Error Control and Loss Functions for the Deep Learning Inversion of Borehole Resistivity Measurements](https://mlsanity.com/paper/2005.08868/) (2020)
- [Stochastic optimal well control in subsurface reservoirs using reinforcement learning](https://mlsanity.com/paper/2207.03456/) (2022)
- [An interactive sequential-decision benchmark from geosteering](https://mlsanity.com/paper/2011.00733/) (2020)

## Built on by (most cited first, 2 of 2)

- [High-Precision Geosteering via Reinforcement Learning and Particle Filters](https://mlsanity.com/paper/2402.06377/) (2024)
- [A Generative-AI Modeling Framework for Explainable Decision Support in Complex Geosteering Scenarios](https://mlsanity.com/paper/2503.08509/) (2025)

## Abstract (arXiv)

Trajectory adjustment decisions throughout the drilling process, called geosteering, affect subsequent choices and information gathering, thus resulting in a coupled sequential decision problem. Previous works on applying decision optimization methods in geosteering rely on greedy optimization or approximate dynamic programming (ADP). Either decision optimization method requires explicit uncertainty and objective function models, making developing decision optimization methods for complex and realistic geosteering environments challenging to impossible. We use the Deep Q-Network (DQN) method, a model-free reinforcement learning (RL) method that learns directly from the decision environment, to optimize geosteering decisions. The expensive computations for RL are handled during the offline training stage. Evaluating DQN needed for real-time decision support takes milliseconds and is faster than the traditional alternatives. Moreover, for two previously published synthetic geosteering scenarios, our results show that RL achieves high-quality outcomes comparable to the quasi-optimal ADP. Yet, the model-free nature of RL means that by replacing the training environment, we can extend it to problems where the solution to ADP is prohibitively expensive to compute. This flexibility will allow applying it to more complex environments and make hybrid versions trained with real data in the future.