mlsanityData to 7 Oct 2026

Risk-averse policies for natural gas futures trading using distributional reinforcement learning

arXiv 2501.04421 ↗First posted 8 Jan 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Financial markets have experienced significant instabilities in recent years, creating unique challenges for trading and increasing interest in risk-averse strategies. Distributional Reinforcement Learning (RL) algorithms, which model the full distribution of returns rather than just expected values, offer a promising approach to managing market uncertainty. This paper investigates this potential by studying the effectiveness of three distributional RL algorithms for natural gas futures trading and exploring their capacity to develop risk-averse policies. Specifically, we analyze the performance and behavior of Categorical Deep Q-Network (C51), Quantile Regression Deep Q-Network (QR-DQN), and Implicit Quantile Network (IQN). To the best of our knowledge, these algorithms have never been applied in a trading context. These policies are compared against five Machine Learning (ML) baselines, using a detailed dataset provided by Predictive Layer SA, a company supplying ML-based strategies for energy trading. The main contributions of this study are as follows. (1) We demonstrate that distributional RL algorithms significantly outperform classical RL methods, with C51 achieving performance improvement of more than 32%. (2) We show that training C51 and IQN to maximize CVaR produces risk-sensitive policies with adjustable risk aversion. Specifically, our ablation studies reveal that lower CVaR confidence levels increase risk aversion, while higher levels decrease it, offering flexible risk management options. In contrast, QR-DQN shows less predictable behavior. These findings emphasize the potential of distributional RL for developing adaptable, risk-averse trading strategies in volatile markets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 115 references that are in the map. First posted on arXiv 8 Jan 2025.

Builds on

41 of 115 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2015Prioritized Experience Replaycited by 1,070
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  10. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  11. 2017Robust Adversarial Reinforcement Learningcited by 439
  12. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
29 more
  1. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  2. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  3. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  4. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  5. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  6. 2017A Tutorial on Kernel Density Estimation and Recent Advancescited by 100
  7. 2014Algorithms for CVaR Optimization in MDPscited by 98
  8. 2021Conservative Offline Distributional Reinforcement Learningcited by 78
  9. 2019Deep Reinforcement Learning for Tradingcited by 73
  10. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  11. 2018Practical Deep Reinforcement Learning Approach for Stock Tradingcited by 49
  12. 2021Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learningcited by 42
  13. 2022Efficient Risk-Averse Reinforcement Learningcited by 41
  14. 2022Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Riskcited by 40
  15. 2018Double Deep Q-Learning for Optimal Executioncited by 39
  16. 2020An Application of Deep Reinforcement Learning to Algorithmic Tradingcited by 30
  17. 2021Deep Hedging of Derivatives Using Reinforcement Learningcited by 30
  18. 2020Improving Robustness via Risk Averse Distributional Reinforcement Learningcited by 28
  19. 2021Risk-Averse Bayes-Adaptive Reinforcement Learningcited by 28
  20. 2022Distributional neural networks for electricity price forecastingcited by 24
  21. 2017QLBS: Q-Learner in the Black-Scholes(-Merton) Worldscited by 23
  22. 2022Gamma and Vega Hedging Using Deep Distributional Reinforcement Learningcited by 15
  23. 2019An intelligent financial portfolio trading strategy using deep Q-learningcited by 14
  24. 2020Forecasting directional movements of stock prices for intraday trading using LSTM and random forestscited by 13
  25. 2020Option Hedging with Risk Averse Reinforcement Learningcited by 13
  26. 2021Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineeringcited by 6
  27. 2022Sliding-Window Normalization to Improve the Performance of Machine-Learning Models for Real-Time Motion Prediction Using Electromyographycited by 6
  28. 2022Risk-Sensitive Policy with Distributional Reinforcement Learningcited by 6
  29. 2024Chain-structured neural architecture search for financial time series forecastingcited by 2

Built on by

Not cited yet.