# Application of Deep Reinforcement Learning to At-the-Money S&P 500 Options Hedging

arXiv: https://arxiv.org/abs/2510.09247 · HTML: https://mlsanity.com/paper/2510.09247/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 10 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Neural networks for option pricing and hedging: a literature review](https://mlsanity.com/paper/1911.05620/) (2019)
- [Deep Hedging of Derivatives Using Reinforcement Learning](https://mlsanity.com/paper/2103.16409/) (2021)
- [QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds](https://mlsanity.com/paper/1712.04609/) (2017)
- [Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions](https://mlsanity.com/paper/2207.07467/) (2022)
- [Deep Hedging: Learning to Remove the Drift under Trading Frictions with Minimal Equivalent Near-Martingale Measures](https://mlsanity.com/paper/2111.07844/) (2021)
- [Robust Risk-Aware Option Hedging](https://mlsanity.com/paper/2303.15216/) (2023)
- [Adversarial Deep Hedging: Learning to Hedge without Price Process Modeling](https://mlsanity.com/paper/2307.13217/) (2023)
- [Hedging with Linear Regressions and Neural Networks](https://mlsanity.com/paper/2004.08891/) (2020)
- [Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information](https://mlsanity.com/paper/2407.21138/) (2024)
- [Deep Reinforcement Learning Algorithms for Option Hedging](https://mlsanity.com/paper/2504.05521/) (2025)

## Built on by (most cited first, 1 of 1)

- [Deep Reinforcement Learning Framework for Diversified Portfolio Management Across Global Equity Markets](https://mlsanity.com/paper/2605.17307/) (2026)

## Abstract (arXiv)

This paper explores the application of deep Q-learning to hedging at-the-money options on the S&P~500 index. We develop an agent based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, trained to simulate hedging decisions without making explicit model assumptions on price dynamics. The agent was trained on historical intraday prices of S&P~500 call options across years 2004--2024, using a single time series of six predictor variables: option price, underlying asset price, moneyness, time to maturity, realized volatility, and current hedge position. A walk-forward procedure was applied for training, which led to nearly 17~years of out-of-sample evaluation. The performance of the deep reinforcement learning (DRL) agent is benchmarked against the Black--Scholes delta-hedging strategy over the same period. We assess both approaches using metrics such as annualized return, volatility, information ratio, and Sharpe ratio. To test the models' adaptability, we performed simulations across varying market conditions and added constraints such as transaction costs and risk-awareness penalties. Our results show that the DRL agent can outperform traditional hedging methods, particularly in volatile or high-cost environments, highlighting its robustness and flexibility in practical trading contexts. While the agent consistently outperforms delta-hedging, its performance deteriorates when the risk-awareness parameter is higher. We also observed that the longer the time interval used for volatility estimation, the more stable the results.