mlsanityData to 14 Oct 2026

Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control

arXiv 2403.04195 ↗First posted 7 Mar 2024cs.LG math.OCcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Changes in demand, various hydrological inputs, and environmental stressors are among the issues that water managers and policymakers face on a regular basis. These concerns have sparked interest in applying different techniques to determine reservoir operation policy decisions. As the resolution of the analysis increases, it becomes more difficult to effectively represent a real-world system using traditional methods such as Dynamic Programming (DP) and Stochastic Dynamic Programming (SDP) for determining the best reservoir operation policy. One of the challenges is the "curse of dimensionality," which means the number of samples needed to estimate an arbitrary function with a given level of accuracy grows exponentially with respect to the number of input variables (i.e., dimensionality) of the function. Deep Reinforcement Learning (DRL) is an intelligent approach to overcome the curses of stochastic optimization problems for reservoir operation policy decisions. To our knowledge, this study is the first attempt that examine various novel DRL continuous-action policy gradient methods (PGMs), including Deep Deterministic Policy Gradients (DDPG), Twin Delayed DDPG (TD3), and two different versions of Soft Actor-Critic (SAC18 and SAC19) for optimizing reservoir operation policy. In this study, multiple DRL techniques were implemented in order to find the optimal operation policy of Folsom Reservoir in California, USA. The reservoir system supplies agricultural, municipal, hydropower, and environmental flow demands and flood control operations to the City of Sacramento. Analysis suggests that the TD3 and SAC are robust to meet the Folsom Reservoir's demands and optimize reservoir operation policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 64 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

5 of 64 references.

References · 5

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  5. 2017Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learningcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Deep Learning Based Simulators for the Phosphorus Removal Process Control in Wastewater Treatment via Deep Reinforcement Learning Algorithms
  2. 2024Improved Long Short-Term Memory-based Wastewater Treatment Simulators for Deep Reinforcement Learning
  3. 2022The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learning
  4. 2025A Practical Introduction to Deep Reinforcement Learning
  5. 2025DRL-Based Medium-Term Planning of Renewable-Integrated Self-Scheduling Cascaded Hydropower to Guide Wholesale Market Participation ↗
  6. 2025Structure-Informed Deep Reinforcement Learning for Inventory Management
  7. 2023Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments
  8. 2025A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report