# Natural-gas storage modelling by deep reinforcement learning

arXiv: https://arxiv.org/abs/2511.02646 · HTML: https://mlsanity.com/paper/2511.02646/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [Towards a fully RL-based Market Simulator](https://mlsanity.com/paper/2110.06829/) (2021)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs](https://mlsanity.com/paper/2302.11835/) (2023)
- [Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling](https://mlsanity.com/paper/2405.02161/) (2024)
- [ADAGE: A generic two-layer framework for adaptive agent based modelling](https://mlsanity.com/paper/2501.09429/) (2025)
- [Tax Credits and Household Behavior: The Roles of Myopic Decision-Making and Liquidity in a Simulated Economy](https://mlsanity.com/paper/2408.10391/) (2024)
- [Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market](https://mlsanity.com/paper/2411.16519/) (2024)

## Abstract (arXiv)

We introduce GasRL, a simulator that couples a calibrated representation of the natural gas market with a model of storage-operator policies trained with deep reinforcement learning (RL). We use it to analyse how optimal stockpile management affects equilibrium prices and the dynamics of demand and supply. We test various RL algorithms and find that Soft Actor Critic (SAC) exhibits superior performance in the GasRL environment: multiple objectives of storage operators - including profitability, robust market clearing and price stabilisation - are successfully achieved. Moreover, the equilibrium price dynamics induced by SAC-derived optimal policies have characteristics, such as volatility and seasonality, that closely match those of real-world prices. Remarkably, this adherence to the historical distribution of prices is obtained without explicitly calibrating the model to price data. We show how the simulator can be used to assess the effects of EU-mandated minimum storage thresholds. We find that such thresholds have a positive effect on market resilience against unanticipated shifts in the distribution of supply shocks. For example, with unusually large shocks, market disruptions are averted more often if a threshold is in place.