# Deep Hedging of Derivatives Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.16409 · HTML: https://mlsanity.com/paper/2103.16409/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 33 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 6 of 6)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Quant GANs: Deep Generation of Financial Time Series](https://mlsanity.com/paper/1907.06673/) (2019)
- [QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds](https://mlsanity.com/paper/1712.04609/) (2017)
- [Style Transfer with Time Series: Generating Synthetic Financial Data](https://mlsanity.com/paper/1906.03232/) (2019)

## Built on by (most cited first, 30 of 30)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Robust Risk-Aware Option Hedging](https://mlsanity.com/paper/2303.15216/) (2023)
- [FuNVol: A Multi-Asset Implied Volatility Market Simulator using Functional Principal Components and Neural SDEs](https://mlsanity.com/paper/2303.00859/) (2023)
- [Markov Decision Processes under Model Uncertainty](https://mlsanity.com/paper/2206.06109/) (2022)
- [Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent](https://mlsanity.com/paper/2407.14486/) (2024)
- [Q-Learning under Finite Model Uncertainty](https://mlsanity.com/paper/2407.04259/) (2024)
- [Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information](https://mlsanity.com/paper/2407.21138/) (2024)
- [Deep Reinforcement Learning Algorithms for Option Hedging](https://mlsanity.com/paper/2504.05521/) (2025)
- [Distributional Adversarial Attacks and Training in Deep Hedging](https://mlsanity.com/paper/2508.14757/) (2025)
- [Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization](https://mlsanity.com/paper/2602.17098/) (2026)
- [Applying Reinforcement Learning to Option Pricing and Hedging](https://mlsanity.com/paper/2310.04336/) (2023)
- [Hedging American Put Options with Deep Reinforcement Learning](https://mlsanity.com/paper/2405.06774/) (2024)
- [Enhancing Black-Scholes Delta Hedging via Deep Learning](https://mlsanity.com/paper/2407.19367/) (2024)
- [Deep Hedging Under Non-Convexity: Limitations and a Case for AlphaZero](https://mlsanity.com/paper/2510.01874/) (2025)
- [Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging](https://mlsanity.com/paper/2510.09247/) (2025)
- [Delta Hedging Liquidity Positions on Automated Market Makers](https://mlsanity.com/paper/2208.03318/) (2022)
- [Reinforcement Learning for Credit Index Option Hedging](https://mlsanity.com/paper/2307.09844/) (2023)
- [Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation](https://mlsanity.com/paper/2402.18149/) (2024)
- [EX-DRL: Hedging Against Heavy Losses with EXtreme Distributional Reinforcement Learning](https://mlsanity.com/paper/2408.12446/) (2024)
- [Solving The Dynamic Volatility Fitting Problem: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2410.11789/) (2024)
- [Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learning](https://mlsanity.com/paper/2412.17908/) (2024)
- [Risk-averse policies for natural gas futures trading using distributional reinforcement learning](https://mlsanity.com/paper/2501.04421/) (2025)
- [Robust and Efficient Deep Hedging via Linearized Objective Neural Network](https://mlsanity.com/paper/2502.17757/) (2025)
- [Risk-Averse Reinforcement Learning with Itakura-Saito Loss](https://mlsanity.com/paper/2505.16925/) (2025)
- [Distributional Reinforcement Learning on Path-dependent Options](https://mlsanity.com/paper/2507.12657/) (2025)
- [A Topological Approach to Parameterizing Deep Hedging Networks](https://mlsanity.com/paper/2510.16938/) (2025)
- [Learning to Hedge Swaptions](https://mlsanity.com/paper/2512.06639/) (2025)
- [NeuralChaos: Optimal Adapted Approximation of Square Integrable Predictable Processes](https://mlsanity.com/paper/2607.14361/) (2026)
- [LiveOption: Evaluating LLM Agents in Structured Option Trading with Nonlinear Payoffs](https://mlsanity.com/paper/2609.33470/) (2026)

## Abstract (arXiv)

This paper shows how reinforcement learning can be used to derive optimal hedging strategies for derivatives when there are transaction costs. The paper illustrates the approach by showing the difference between using delta hedging and optimal hedging for a short position in a call option when the objective is to minimize a function equal to the mean hedging cost plus a constant times the standard deviation of the hedging cost. Two situations are considered. In the first, the asset price follows a geometric Brownian motion. In the second, the asset price follows a stochastic volatility process. The paper extends the basic reinforcement learning approach in a number of ways. First, it uses two different Q-functions so that both the expected value of the cost and the expected value of the square of the cost are tracked for different state/action combinations. This approach increases the range of objective functions that can be used. Second, it uses a learning algorithm that allows for continuous state and action space. Third, it compares the accounting P&L approach (where the hedged position is valued at each step) and the cash flow approach (where cash inflows and outflows are used). We find that a hybrid approach involving the use of an accounting P&L approach that incorporates a relatively simple valuation model works well. The valuation model does not have to correspond to the process assumed for the underlying asset price.