# QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds

arXiv: https://arxiv.org/abs/1712.04609 · HTML: https://mlsanity.com/paper/1712.04609/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2017.

## Built on by (most cited first, 23 of 23)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Deep Hedging of Derivatives Using Reinforcement Learning](https://mlsanity.com/paper/2103.16409/) (2021)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Markov Decision Processes under Model Uncertainty](https://mlsanity.com/paper/2206.06109/) (2022)
- [Hedging of Financial Derivative Contracts via Monte Carlo Tree Search](https://mlsanity.com/paper/2102.06274/) (2021)
- [Deep Reinforcement Learning Algorithms for Option Hedging](https://mlsanity.com/paper/2504.05521/) (2025)
- [Reinforcement Learning for Option Hedging: Static Implied-Volatility Fit versus Shortfall-Aware Performance](https://mlsanity.com/paper/2601.01709/) (2026)
- [Applying Reinforcement Learning to Option Pricing and Hedging](https://mlsanity.com/paper/2310.04336/) (2023)
- [Hedging American Put Options with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.06774/) (2024)
- [Enhancing Black-Scholes Delta Hedging via Deep Learning](https://mlsanity.com/paper/2407.19367/) (2024)
- [Hedging and Pricing Structured Products Featuring Multiple Underlying Assets](https://mlsanity.com/paper/2411.01121/) (2024)
- [Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging](https://mlsanity.com/paper/2510.09247/) (2025)
- [Can we imitate the principal investor's behavior to learn option price?](https://mlsanity.com/paper/2105.11376/) (2021)
- [Reinforcement learning for options on target volatility funds](https://mlsanity.com/paper/2112.01841/) (2021)
- [RLOP: RL Methods in Option Pricing from a Mathematical Perspective](https://mlsanity.com/paper/2205.05600/) (2022)
- [Reinforcement Learning for Credit Index Option Hedging](https://mlsanity.com/paper/2307.09844/) (2023)
- [EX-DRL: Hedging Against Heavy Losses with EXtreme Distributional Reinforcement Learning](https://mlsanity.com/paper/2408.12446/) (2024)
- [Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learning](https://mlsanity.com/paper/2412.17908/) (2024)
- [Risk-averse policies for natural gas futures trading using distributional reinforcement learning](https://mlsanity.com/paper/2501.04421/) (2025)
- [Risk-Averse Reinforcement Learning with Itakura-Saito Loss](https://mlsanity.com/paper/2505.16925/) (2025)
- [Constrained deep learning for pricing and hedging european options in incomplete markets](https://mlsanity.com/paper/2511.20837/) (2025)
- [Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning](https://mlsanity.com/paper/2603.06587/) (2026)
- [Climate-Dyna Deep Hedging for XVAs: Model-Based Reinforcement Learning, Residual Climate HVA, and Hedge-Instrument Discovery](https://mlsanity.com/paper/2608.01208/) (2026)

## Abstract (arXiv)

This paper presents a discrete-time option pricing model that is rooted in Reinforcement Learning (RL), and more specifically in the famous Q-Learning method of RL. We construct a risk-adjusted Markov Decision Process for a discrete-time version of the classical Black-Scholes-Merton (BSM) model, where the option price is an optimal Q-function, while the optimal hedge is a second argument of this optimal Q-function, so that both the price and hedge are parts of the same formula. Pricing is done by learning to dynamically optimize risk-adjusted returns for an option replicating portfolio, as in the Markowitz portfolio theory. Using Q-Learning and related methods, once created in a parametric setting, the model is able to go model-free and learn to price and hedge an option directly from data, and without an explicit model of the world. This suggests that RL may provide efficient data-driven and model-free methods for optimal pricing and hedging of options, once we depart from the academic continuous-time limit, and vice versa, option pricing methods developed in Mathematical Finance may be viewed as special cases of model-based Reinforcement Learning. Further, due to simplicity and tractability of our model which only needs basic linear algebra (plus Monte Carlo simulation, if we work with synthetic data), and its close relation to the original BSM model, we suggest that our model could be used for benchmarking of different RL algorithms for financial trading applications