# The QLBS Q-Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios

arXiv: https://arxiv.org/abs/1801.06077 · HTML: https://mlsanity.com/paper/1801.06077/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2018.

## Built on by (most cited first, 8 of 8)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [A Survey on Deep Reinforcement Learning for Data Processing and Analytics](https://mlsanity.com/paper/2108.04526/) (2021)
- [Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information](https://mlsanity.com/paper/2407.21138/) (2024)
- [Reinforcement Learning for Option Hedging: Static Implied-Volatility Fit versus Shortfall-Aware Performance](https://mlsanity.com/paper/2601.01709/) (2026)
- [Applying Reinforcement Learning to Option Pricing and Hedging](https://mlsanity.com/paper/2310.04336/) (2023)
- [Enhancing Black-Scholes Delta Hedging via Deep Learning](https://mlsanity.com/paper/2407.19367/) (2024)
- [Reinforcement Learning for Credit Index Option Hedging](https://mlsanity.com/paper/2307.09844/) (2023)
- [Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning](https://mlsanity.com/paper/2603.06587/) (2026)

## Abstract (arXiv)

The QLBS model is a discrete-time option hedging and pricing model that is based on Dynamic Programming (DP) and Reinforcement Learning (RL). It combines the famous Q-Learning method for RL with the Black-Scholes (-Merton) model's idea of reducing the problem of option pricing and hedging to the problem of optimal rebalancing of a dynamic replicating portfolio for the option, which is made of a stock and cash. Here we expand on several NuQLear (Numerical Q-Learning) topics with the QLBS model. First, we investigate the performance of Fitted Q Iteration for a RL (data-driven) solution to the model, and benchmark it versus a DP (model-based) solution, as well as versus the BSM model. Second, we develop an Inverse Reinforcement Learning (IRL) setting for the model, where we only observe prices and actions (re-hedges) taken by a trader, but not rewards. Third, we outline how the QLBS model can be used for pricing portfolios of options, rather than a single option in isolation, thus providing its own, data-driven and model independent solution to the (in)famous volatility smile problem of the Black-Scholes model.